Hot answers tagged

57 votes
Accepted

How to set batch_size, steps_per epoch, and validation steps?

batch_size determines the number of samples in each mini batch. Its maximum is the number of all samples, which makes gradient descent accurate, the loss will decrease towards the minimum if the ...
Silpion's user avatar
  • 761
35 votes
Accepted

What is the difference between upsampling and bi-linear upsampling in a CNN?

In the context of image processing, upsampling is a technique for increasing the size of an image. For example, say you have an image with a height and width of $64$ pixels each (totaling $64 \times ...
Djib2011's user avatar
  • 7,948
32 votes

What's the difference between Attention vs Self-Attention? What problems does each other solve that the other can't?

Here's the list of difference that I know about attention (AT) and self-attention (SA). In neural networks you have inputs before layers, activations (outputs) of the layers and in RNN you have ...
artoby's user avatar
  • 421
31 votes
Accepted

In CNN, why do we increase the number of filters in deeper Convolution layers for complex images?

For this you need to understand what filters actually do. Every layer of filters is there to capture patterns. For example, the first layer of filters captures patterns like edges, corners, dots etc. ...
ashukid's user avatar
  • 877
28 votes

Why do convolutional neural networks work?

Actually I guess the question is a bit broad! Anyway. Understanding Convolution Nets What is learned in ConvNets tries to minimize the cost function to ...
Green Falcon's user avatar
26 votes
Accepted

What is a channel in a CNN?

Let's assume that we are talking about 2D convolutions applied on images. In a grayscale image, the data is a matrix of dimensions $w \times h$, where $w$ is the width of the image and $h$ is its ...
noe's user avatar
  • 25.7k
25 votes

What's the difference between Attention vs Self-Attention? What problems does each other solve that the other can't?

Let me try to keep it more intuitive and less mathematical Prior to 2014, RNNs used to perform badly if the sequence was beyond a certain size. After all RNNs encode all steps in the sequence and give ...
Allohvk's user avatar
  • 888
18 votes

Why do convolutional neural networks work?

ConvNets work because they exploit feature locality. They do it at different granularities, therefore being able to model hierarchically higher level features. They are translation invariant thanks to ...
noe's user avatar
  • 25.7k
17 votes
Accepted

back propagation in CNN

A convolution employs a weight sharing principle which will complicate the mathematics significantly but let's try to get through the weeds. I am drawing most of my explanation from this source. ...
JahKnows's user avatar
  • 8,856
15 votes
Accepted

Updating the weights of the filters in a CNN

In a normal neural network, each neuron has its own weight. This is not correct. Every connection between neurons has its own weight. In a fully connected network each neuron will be associated with ...
Imran's user avatar
  • 2,381
15 votes
Accepted

How to add non-image features along side images as the input of CNNs

My solution is like your first recommendation, but with slight changes. Construct your convolutional layers and stack them till the flatten-layer. This network should be fed with the image data. Flat ...
Green Falcon's user avatar
15 votes

In CNN, why do we increase the number of filters in deeper Convolution layers for complex images?

The higher the number of filters, the higher the number of abstractions that your Network is able to extract from image data. The reason why the number of filters is generally ascending is that at the ...
Leevo's user avatar
  • 6,185
14 votes
Accepted

How to get predicted class labels in convolution neural network?

What you have are predicted class probabilities. Since you are doing binary classification, each output is the probability of the first class for that test example. To convert these to class labels ...
Imran's user avatar
  • 2,381
14 votes
Accepted

When are weights updated in CNN?

Whenever you train the network using batch means that you have chosen to train using batch gradient descent. There are three variants for gradient descent algorithm: Gradient Descent Stochastic ...
Green Falcon's user avatar
14 votes

What are the differences between Convolutional1D, Convolutional2D, and Convolutional3D?

Conv1D is used for input signals which are similar to the voice. By employing them you can find patterns across the signal. For instance, you have a voice signal and you have a convolutional layer. ...
Green Falcon's user avatar
14 votes

Why should I understand AI architectures?

You are right that you actually do not need to know the architectures if you just want to apply them. But there are to reasons why it would be good to understand the architecture. Models often do not ...
MachineLearner's user avatar
13 votes
Accepted

CNN - How does backpropagation with weight-sharing work exactly?

I think you're misunderstanding what "weight sharing" means here. A convolutional layer is generally comprised of many "filters", which are usually 2x2 or 3x3. These filters are applied in a "sliding ...
David Marx's user avatar
  • 3,228
13 votes
Accepted

What is the input size of Alex net

I guess it has been a mistake. Take a look at here. The other author's were Ilya Sutskever and Geoffrey Hinton. So, AlexNet input starts with 227 by 227 by 3 images. And if you read the paper, the ...
Green Falcon's user avatar
13 votes
Accepted

Effect of NOT changing filter weights of CNN during backprop

By not changing the weights of the convolutional layers of a CNN, you are essentially feeding your classifier (the fully connected layer) random features (i.e. not the optimal features for the ...
Djib2011's user avatar
  • 7,948
13 votes
Accepted

What are the differences between Convolutional1D, Convolutional2D, and Convolutional3D?

The only difference is the dimensionality of the input space. The input for a convolutional layer has the following shape: input_shape = (batch_size,input_dims,channels) Input shape for conv1D: (...
ignatius's user avatar
  • 1,658
13 votes
Accepted

How to use a dataset with only one category of data

The Model learns to match the weights as per the image and feedback from label data. If you will feed a few Image classes as "Not Cat", it will learn to classify similar features as "...
10xAI's user avatar
  • 5,574
12 votes
Accepted

Determining size of FC layer after Conv layer in PyTorch

Hello and welcome to Stack Exchange! The answer to your question is quite simple: you did not use the correct formula. The formula you used is (assuming we are working with square inputs) $$ W'=\...
RaptorDotCpp's user avatar
12 votes
Accepted

Why must a CNN have a fixed input size?

I think the answer to this question is weight sharing in convolutional layers, which you don't have in fully-connected ones. In convolutional layers you only train the kernel, which is then convolved ...
matthiaw91's user avatar
  • 1,535
12 votes
Accepted

Understanding how convolutional layers work

What are the filters? A filter/kernel is a set of learnable weights which are learned using the backpropagation algorithm. You can think of each filter as storing a single template/pattern. When you ...
Akshay Sehgal's user avatar
11 votes

Is there any proven disadvantage of transfer learning for CNNs?

Based on my experience, not just for ImageNet, if you have enough data it's better to train your network from scratch. There are numerous reasons that I can explain why. First of all, I don't know ...
Green Falcon's user avatar
11 votes

How to prepare the varied size input in CNN prediction

Conventionally, when dealing with images of different sizes in CNN(which happens very often in real world problems), we resize the images to the size of the smallest images with the help of any image ...
Amruth Lakkavaram's user avatar
11 votes
Accepted

What's the principal difference between ANN,RNN,DNN and CNN?

Welcome to DS StackExchange. I'll go through your list: ANN (Artificial Neural Network): it's a very broad term that encompasses any form of Deep Learning model. All the others you listed are some ...
Leevo's user avatar
  • 6,185
10 votes

CNN - How does backpropagation with weight-sharing work exactly?

I'm not sure if you can change accepted answers, but since the only answer to your question on back propagation is one about forward propagation, I decided to give it a go. Essentially, you treat the ...
longbowrocks's user avatar
9 votes
Accepted

Are CNNs insensitive to rotations and shifts in images?

If you use max-pooling layers, they may be insensetive to small shifts but not that much. If you want your network to be able to be invariant to transformations, ...
Green Falcon's user avatar
9 votes
Accepted

Batch normalization vs batch size

By increasing batch size your steps can be more accurate because your sampling will be closer to the real population. If you increase the size of batch, your batch normalisation can have better ...
Green Falcon's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible