Questions tagged [neural-network]
Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.
4,376
questions
2
votes
2
answers
2k
views
Keras models break when I add batch normalization
I'm creating the model for a DDPG agent (keras-rl version) but i'm having some trouble with errors whenever I try adding in batch normalization in the first of two networks.
Here is the creation ...
- 23
0
votes
1
answer
133
views
Neural Network for regression with one dependent and one independent variable
I am trying to make a simple neural network with one dependent and one independent variable. Could you maybe give me a tutorial or help me with the implementation of a neural network with one ...
2
votes
1
answer
1k
views
Is there a way to get y_pred values from saved Keras model?
I have a Keras model saved in a .h5 file. As you know there are a y_pred and a y_act that ...
- 1,147
3
votes
3
answers
860
views
predicted bounding boxes that stretch beyond grid cell (Andrew NG CNN course)?
I was following Pr. Andrew Ng course on Course about Convolutional neural network and I have a doubt regarding one of the points he mentions in the ...
- 931
1
vote
2
answers
105
views
"Understanding Machine Learning: From Theory to Algorithms" the universal approximation theorem
I'm readying on "Understanding Machine Learning: From Theory to Algorithms" the Universal approximation theorem:
..."Networks are universal approximators. That is, for every fixed precision ...
- 11
2
votes
1
answer
841
views
Why use different variations of Softmax in training and validation for neural networks with Pytorch?
Specifically, I'm working on a modeling project, and I see someone else's code that looks like
...
- 123
1
vote
1
answer
545
views
How to transform stock data for LSTM-based neural network
I am trying to classify stock returns using an LSTM-based neural network.
I would like to use closing price and volume as features (see below), but am unsure of whether I need to transform these (e.g....
- 53
1
vote
0
answers
31
views
Using neural nets with just one input variable (without response/feature) [closed]
Hi i am using following data from R library ftnonpar:
...
- 11
1
vote
1
answer
79
views
Neural network is getting partially trained
So I am writing my own neural network library using back-propagation as my training algorithm. Everything seems fine the error is getting decreased more and more at each iteration however when I am ...
0
votes
1
answer
88
views
Does adding of many FC layers during re-training increase the model size ? Are there any ways to optimize the size of model?
I am re-training a pretrained model VGG16.
In the last layers, im using two FC layers of size 2048 each, with dropout=0.5.
When I saved the model, the size of the ...
0
votes
1
answer
3k
views
Improve the accuracy for multi-label classification (Scikit-learn, Keras)
I am going to train machine learning models that assign certain tags to a paragraph describing an activity. In my database, for a give paragraph of description (X), there are several corresponding ...
1
vote
1
answer
279
views
Class Imbalance Problem even after Balancing Data
So I am training a neural network on a binary classification problem and my Case (1) and Controls (0) were imbalanced so I oversampled my cases so that that the training set was 0.5053 made up of ...
- 45
0
votes
0
answers
41
views
How to design my own keras layer?
I am implementing the paper Perceptual GAN for small object detection. The design is described by the picture given below. I need to design my own keras layer. I have described my code below:
The ...
- 141
2
votes
0
answers
27
views
Which one of these is the most efficient way to model training data for a neural network that will play a snake-like game?
I am building an AI using a neural network that will play Tron against a human player. The game consists of a board with fixed width and height where each player can move at any direction (except for ...
- 41
0
votes
1
answer
273
views
How do I convert a summation equation to a vector equation (backpropagation)?
$$a_j^l=\sigma(\sum_{k} {w_j}_k^l {a}_k^{l-1}+b_j^l)$$
$$a^l=\sigma( w^l {a}^{l-1}+b^l)$$
In a resource I have been reading, the above equations describe the activation of a neurone. They have the ...
- 451
0
votes
1
answer
69
views
Learn the evaluation matrix for hire a job candidate
Let us assume that we have a certain number of features that are weighted with some parameters . The features could be the different skills that belongs to a job candidate applying for a job ...
- 21
0
votes
1
answer
248
views
Confused with the derivation of the gradient descent update rule
I have been going over some theory for gradient descent. The source I am looking at said that the change in cost can be described by the following equation:
$$∆C=∇C∙∆w$$
where $∇C$ is the gradient ...
- 451
4
votes
3
answers
7k
views
Input contains NaN, infinity or a value too large for dtype('float32'). Without ruining Dataset
I have a datasheet that is all in Binary but sometimes there are missing cases. In one of the inputs, there is a blank, I don't want to completely remove the whole column because some of the rows have ...
- 41
2
votes
0
answers
307
views
How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]
I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material.
When discussing the training algorithm for a perceptron, the author states that looping ...
1
vote
0
answers
55
views
Examples of using GANs to sort numbers?
Does anyone know of any publicly available GAN implementations to sort numbers with? As in, the input to the generator is an unordered sequence of numbers, and the goal of the generator is to output ...
- 11
0
votes
2
answers
1k
views
What can be the best dropout value and the FC layer for good accurate predictions?
I am retraining the pre-trained model VGG16 in the last FC layers.
I used the below function .
what can be the best combination of FC layers and dropout values for the best predictions. ?
1
vote
1
answer
739
views
What are h(t-1) and c(t-1) for the first LSTM cell?
I know in a LSTM chain you should connect the h(t) of the previous cell to the h(t+1) of the next cell, and doing so for c(t). But what about the first cell? What does it get as h(t-1) and c(t-1)?
I ...
- 1,270
1
vote
0
answers
93
views
Calculate loss from intermediate layer
Say I have an intermediate embedding layer (say at first layer) in a neural network of four layers. A desired property of the embedding matrix is that the sparser the better. Say the output from ...
- 775
2
votes
1
answer
382
views
Layer notation for feed forward neural networks
Apologies in advance, for I have a fairly rudimentary question on the notations for studying Feed-Forward Neural Networks. Here is a nice schematic taken from this blog-post.
Here $x_i = f_i(W_i \...
- 133
1
vote
0
answers
43
views
Why does NAG cause unstable validation loss?
I'm building a neural network for a classification problem. When playing around with some hyperparameters, I was surprised to see that using Nesterov's Accelerated Gradient instead of vanilla SGD ...
0
votes
0
answers
48
views
Constrained Deep Learning
What are the ways to perform constrained optimization of weights in deep model? I am working on deep similarity learning, and many methods are based on constrained optimization; the vast majority of ...
- 80
2
votes
2
answers
4k
views
What is the difference between Concatenate() and concatenate() layers in Keras / TensorFlow 2.0?
I am learning TensorFlow 2.0, whose layer functions are based on Keras. What is the difference between the Concatenate() and ...
- 6,225
1
vote
1
answer
463
views
Making sense of indices in 2D convolution operations in convolutional neural networks
Referring to the answer here: https://www.quora.com/Why-are-convolutional-nets-called-so-when-they-are-actually-doing-correlations, the equation for a discrete 2D convolution is specified as:
$$C(x,y)...
- 299
3
votes
1
answer
7k
views
How to use decode_predictions() for non-Imagenet models..?
I know that decode_predictions() works for only imagenet dataset(1000 classes) for the models like VGG16 etc.
But condiser my scenario.
My Scenario:
I used vgg16 pretrained model, and added my own ...
0
votes
1
answer
49
views
What does this expression from gradient descent mean?
I am looking over some neural network theory and came across this equation, coupled with this description (gradient descent ball-valley analogy):
''let's think about what happens when we move the ...
- 451
0
votes
1
answer
565
views
CNN Architecture for Multiple Instance Learning
I have a binary classification problem where I have a bag of documents (image files) that I need to classify - the bag that is, not the individual document. However, a bag can have a different number ...
2
votes
1
answer
389
views
Predicting word from a set of words
My task is to predict relevant words based on a short description of an idea. for example "SQL is a domain-specific language used in programming and designed for managing data held in a relational ...
- 221
1
vote
1
answer
371
views
Improving Accuracy of the Deep Learning Model
In my current project, I have only 647 rows (500 for training and 147 for testing) and I have applied the Keras Sequential model using the following code:
...
- 123
4
votes
3
answers
2k
views
Different learning rates for each dimension
I have been thinking about why normalization and scaling are done for each feature in the basic context of gradient descent.
One thing that got me wondering is that we use a pre-defined set of ...
0
votes
1
answer
293
views
Does it make sense to train an Autoencoder for Dimensionality Reduction using Mini-Batch Gradient Descent?
I want to reduce the dimensionality of a dataset using a stacked Autoencoder. The size of the dataset and the computing power at my disposal make it very difficult to train the Network using simple, ...
- 6,225
2
votes
1
answer
225
views
Temporal difference learning with a neural network
Suppose I would like to train a value network $v$ via TD(0).
So my TD target for a time step $t$ equals:
$$R_{t+1} + \gamma v(s_{t+1})$$
If I understand correctly, I just need to use mean squared ...
- 23
0
votes
1
answer
303
views
while re-training a pre-trained model, I'm facing this issue RuntimeError: You must compile your model before using it
model summary:
RuntimeError: You must compile your model before using it.
It says that the model needs to be compiled. But as far i know, if i compile a model, all the previous trained data will be ...
1
vote
0
answers
80
views
Having problem in back propagation part for dimension
I was trying to build a neural network with single hidden layer from scratch. In back propagation part some problems have raised.
For calculating gradient of loss function with respect to weight in ...
- 123
0
votes
1
answer
125
views
Issues related to the code for ROI pooling from the feature map
I am trying to do ROI pooling on the feature map obtained from the VGG layers but I don't know how to code this layers. Can anybody help me out?
Here is my VGG layers:
...
- 141
2
votes
1
answer
187
views
Neural Network backprop formula - Matrix dimensions won't match?
I want to start by taking an example for a normal neural network with 2 input nodes, 3 hidden nodes and 2 output nodes. Let the weights between input and hidden nodes are $W_i{_j}$ (2x3) and weights ...
- 420
1
vote
1
answer
244
views
Neural Network - exercise
I am currently learning the concept of neural networks by myself. I am working with a very good pdf from http://neuralnetworksanddeeplearning.com/chap1.html
I also did a few exercises but there is ...
- 113
2
votes
1
answer
2k
views
0
votes
2
answers
2k
views
Which performs better in time series forecasting, LSTM or SVR?
I have run LSTM and SVR models on various datasets having sample values in the range of 1-4000 and the MAPE obtained in SVR was consistently lesser than that obtained through LSTM. I was told the ...
- 1
0
votes
0
answers
277
views
Can Facebook's Prophet be considered a “type” of AI? Like and CNN or RNN
I'm facing a problem to determine if the model Prophet developed by Facebook team is some kind of Neural Network or similar.
Someone has an answer for it?
In Github repository has the following ...
- 116
1
vote
0
answers
25
views
Visualizing general adversarial network
I am working on a DC GAN model using my own data set. How can I visualize (see output) of the GAN generator to see how my network is working (replicating the training data)?
Here is my code:
...
- 141
0
votes
1
answer
2k
views
Resize instead of transposed convolutions
I'm trying to build a decoder version of ResNet, i.e. one that goes from the prelogits layer and attempts to recreate the image. I can get it working by using transposed convolutions, but the quality ...
- 101
9
votes
2
answers
9k
views
Is Faster RCNN the same thing as VGG-16, RESNET-50, etc... or not?
My understanding is that Faster RCNN is an architecture for performing object detection. It finds objects in an image and classifies them. My understanding is also that VGG-16, RESNET-50, etc... also ...
- 91
2
votes
2
answers
69
views
Does partial transfer learning require a lot of computer power?
I want to be sure my understanding of the problem is correct.
I want to do image classification and current state of the art in my field is achieved by transfer learning with VGG16.
Since image on ...
- 155
1
vote
2
answers
52
views
Neural network got a lucky guess. Can it be trusted?
Say you come across a loss curve as shown below. At which loss should you trust the model? The initial lucky guess or after it has stabilized?
And more importantly, why?
- 163
1
vote
0
answers
333
views
Error while trying to do hyperparameter tuning using hyperas
I am getting a syntax error while using hyperas and am not sure why.
My code:
...
- 107