Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

Validation error is slightly lower than training error, but only for some initial conditions

Fair warning, I'm new to this field, so my process may be odd. Any advice is appreciated. I am currently training a model to reproduce some DFT (density functional theory) data. I have been doing ...
0
votes
1answer
16 views

What is the difference between majority vote, and greedy action in ensembling?

I read some stuff about majority vote and greedy action in ensembling, however, they kind of sound similar, but also different. What is the real difference between those two? Thanks for your help!
2
votes
1answer
31 views

What are the exact differences between Deep Learning, Deep Neural Networks, Artificial Neural Networks and further terms?

After having read some theory I am getting a bit confused about the following terms: Deep Learning Deep Neural Network Artificial Neural Network Feedforward Neural Network So, what seems clear to me ...
2
votes
2answers
35 views

Two different cost functions for neural networks, how they can give the same result?

One is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}\Big[y_{k}^{i}\log\big((h_{\theta}(x^{i}))_k\big)+(1-y_{k}^{i})\log\big(1-(h_{\theta}(x^{i}))_k\big)\Big]$$ The other one is: $$J=-\frac{1}{m}\sum_{...
3
votes
1answer
17 views

Multi-Head attention mechanism in transformer and need of feed forward neural network

After reading the paper, Attention is all you need, I have two questions: 1. What is the need of a multi-head attention mechanism? The paper says that: "Multi-head attention allows the model to ...
0
votes
1answer
8 views

Derivative of activation function used in gradient descent algorithms

Why is it necessary to calculate the derivative of activation functions while updating model( regression or NN) parameters? Why is the constant gradient of linear functions considered as a ...
0
votes
0answers
8 views

Thresholding in intermediate layer using Gumbel Softmax

In a neural network, for an intermediate layer, I need to threshold the output. The output of each neuron in the layer is a real value, but I need to binarize it (to 0 or 1). But with hard ...
0
votes
1answer
19 views

Deep Neural Network and Activation Function

I want to know why we need Activation function in DNN hidden layers. I know a bit, like it will help us in, Increasing model complexity and introduce non-linearity Avoiding Gradient Vanishing ...
2
votes
2answers
30 views

In CNN, why do we increase the number of filters in deeper Convolution layers for complex images?

I have been doing this online course Introduction to TensorFlow for AI, ML and DL. Here in one part, they were showing a CNN model for classifying human and horses. In this model, the first ...
1
vote
0answers
15 views

Keras models break when I add batch normalization

I'm creating the model for a DDPG agent (keras-rl version) but i'm having some trouble with errors whenever I try adding in batch normalization in the first of two networks. Here is the creation ...
0
votes
1answer
40 views

Neural Network for regression with one dependent and one independent variable

I am trying to make a simple neural network with one dependent and one independent variable. Could you maybe give me a tutorial or help me with the implementation of a neural network with one ...
1
vote
1answer
24 views

Is there a way to get y_pred values from saved keras model?

I have a saved Keras model that saved in .h5 file, as you know there are a y_pred and y_act ...
2
votes
2answers
38 views

predicted bounding boxes that stretch beyond grid cell (Andrew NG CNN course)?

I was following Pr. Andrew Ng course on Course about Convolutional neural network and I have a doubt regarding one of the points he mentions in the ...
1
vote
2answers
32 views

“Understanding Machine Learning: From Theory to Algorithms” the universal approximation theorem

I'm readying on "Understanding Machine Learning: From Theory to Algorithms" the Universal approximation theorem: ..."Networks are universal approximators. That is, for every fixed precision ...
2
votes
1answer
15 views

Why use different variations of Softmax in training and validation for neural networks with Pytorch?

Specifically, I'm working on a modeling project, and I see someone else's code that looks like ...
1
vote
1answer
26 views

How to transform stock data for LSTM-based neural network

I am trying to classify stock returns using an LSTM-based neural network. I would like to use closing price and volume as features (see below), but am unsure of whether I need to transform these (e.g....
0
votes
0answers
9 views

Why/when is someone resetting a recurrent neural network necessary?

This is my first time implementing a recurrent neural network, and I'm confused as to why resetting the node activations is necessary. When is it necessary to reset node activations? Specifically, ...
1
vote
0answers
27 views

Using neural nets with just one input variable (without response/feature) [on hold]

Hi i am using following data from R library ftnonpar: ...
1
vote
1answer
30 views

Neural network is getting partially trained

So I am writing my own neural network library using back-propagation as my training algorithm. Everything seems fine the error is getting decreased more and more at each iteration however when I am ...
0
votes
1answer
23 views

Does adding of many FC layers during re-training increase the model size ? Are there any ways to optimize the size of model?

I am re-training a pretrained model VGG16. In the last layers, im using two FC layers of size 2048 each, with dropout=0.5. When i saved the model, The size of the model was found to be 2 GB (which ...
0
votes
0answers
14 views

what is the best model in neural network to predict ordered objects list?

I have some fruits with properties and I would like to predict the order of the input fruits using neural network. For example let say I have 3 fruits: orange, apple, banana. And the properties of ...
0
votes
0answers
8 views

Neural Net to represent Multivariate Distribution

I want to use a neural net to learn an arbitrary multivariate distribution (say, 3 variables). If this was in one variable, I would make the neural net monotonically increasing to have it represent ...
0
votes
1answer
36 views

Improve the accuracy for multi-label classification (Scikit-learn, Keras)

I am going to train machine learning models that assign certain tags to a paragraph describing an activity. In my database, for a give paragraph of description (X), there are several corresponding ...
1
vote
1answer
28 views

Class Imbalence Problem even after Balancing Data

So I am training a neural network on a binary classification problem and my Case (1) and Controls (0) were imbalanced so I oversampled my cases so that that the training set was 0.5053 made up of ...
0
votes
0answers
14 views

Why gpt-2 could apply to other tasks without fine-tune?

Language Models are Unsupervised Multitask Learners https://github.com/openai/gpt-2
0
votes
0answers
21 views

How to design my own keras layer?

I am implementing the paper Perceptual GAN for small object detection. The design is described by the picture given below. I need to design my own keras layer. I have described my code below: The ...
2
votes
0answers
21 views

Which one of these is the most efficient way to model training data for a neural network that will play a snake-like game?

I am building an AI using a neural network that will play Tron against a human player. The game consists of a board with fixed width and height where each player can move at any direction (except for ...
0
votes
1answer
14 views

How do I convert a summation equation to a vector equation (backpropagation)?

$$a_j^l=\sigma(\sum_{k} {w_j}_k^l {a}_k^{l-1}+b_j^l)$$ $$a^l=\sigma( w^l {a}^{l-1}+b^l)$$ In a resource I have been reading, the above equations describe the activation of a neurone. They have the ...
0
votes
1answer
16 views

Learn the evaluation matrix for hire a job candidate

Let us assume that we have a certain number of features that are weighted with some parameters . The features could be the different skills that belongs to a job candidate applying for a job ...
0
votes
1answer
13 views

Confused with the derivation of the gradient descent update rule

I have been going over some theory for gradient descent. The source I am looking at said that the change in cost can be described by the following equation: $$∆C=∇C∙∆w$$ where $∇C$ is the gradient ...
2
votes
1answer
24 views

Input contains NaN, infinity or a value too large for dtype('float32'). Without ruining Dataset

I have a datasheet that is all in Binary but sometimes there are missing cases. In one of the inputs, there is a blank, I don't want to completely remove the whole column because some of the rows have ...
2
votes
0answers
23 views

How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]

I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material. When discussing the training algorithm for a perceptron, the author states that looping ...
1
vote
0answers
14 views

Examples of using GANs to sort numbers?

Does anyone know of any publicly available GAN implementations to sort numbers with? As in, the input to the generator is an unordered sequence of numbers, and the goal of the generator is to output ...
0
votes
2answers
24 views

What can be the best dropout value and the FC layer for good accurate predictions?

I am retraining the pre-trained model VGG16 in the last FC layers. I used the below function . what can be the best combination of FC layers and dropout values for the best predictions. ?
0
votes
0answers
11 views

Multiple inputs to Multiple Neural network in parallel in Keras or Pytorch

I want to make a deep network as shown in the image. I want each 'network 1' to look at the specific part of the input and I don't want to divide my input beforehand in chunks. Is there any such ...
0
votes
1answer
35 views

What are h(t-1) and c(t-1) for the first LSTM cell?

I know in a LSTM chain you should connect the h(t) of the previous cell to the h(t+1) of the next cell, and doing so for c(t). But what about the first cell? What does it get as h(t-1) and c(t-1)? I ...
1
vote
0answers
14 views

Calculate loss from intermediate layer

Say I have an intermediate embedding layer (say at first layer) in a neural network of four layers. A desired property of the embedding matrix is that the sparser the better. Say the output from ...
2
votes
1answer
34 views

Layer notation for feed forward neural networks

Apologies in advance, for I have a fairly rudimentary question on the notations for studying Feed-Forward Neural Networks. Here is a nice schematic taken from this blog-post. Here $x_i = f_i(W_i \...
1
vote
0answers
7 views

Why does NAG cause unstable validation loss?

I'm building a neural network for a classification problem. When playing around with some hyperparameters, I was surprised to see that using Nesterov's Accelerated Gradient instead of vanilla SGD ...
0
votes
0answers
3 views

How calculate computation time for each part of the network

I want to report how much times it takes to compute each specific part of the network in a batch (forward and backward time). For example, in this paper they've reported RNN, softmax, and optimization ...
0
votes
0answers
26 views

Constrained Deep Learning

What are the ways to perform constrained optimization of weights in deep model? I am working on deep similarity learning, and many methods are based on constrained optimization; the vast majority of ...
0
votes
2answers
18 views

What is the difference between Concatenate() and concatenate() layers in Keras / TensorFlow 2.0?

I am learning TensorFlow 2.0, whose layer functions are based on Keras. What is the difference between the Concatenate() and ...
1
vote
1answer
24 views

Making sense of indices in 2D convolution operations in convolutional neural networks

Referring to the answer here: https://www.quora.com/Why-are-convolutional-nets-called-so-when-they-are-actually-doing-correlations, the equation for a discrete 2D convolution is specified as: $$C(x,y)...
0
votes
1answer
13 views

How to use decode_predictions() for non-Imagenet models..?

I know that decode_predictions() works for only imagenet dataset(1000 classes) for the models like VGG16 etc. But condiser my scenario. My Scenario: I used vgg16 pretrained model, and added my own ...
0
votes
1answer
16 views

What does this expression from gradient descent mean?

I am looking over some neural network theory and came across this equation, coupled with this description (gradient descent ball-valley analogy): ''let's think about what happens when we move the ...
0
votes
0answers
30 views

2019 - Bleeding edge Reinforcement Learning techniques?

I've built an RL agent using the following: Full Rainbow: Double Q-Learning (allow target network to rate the Q-score of the action selected by online network, use this score as a TD target) ...
0
votes
0answers
14 views

How LSTM compare which information is important or not?

I am interested to know, if I have scaled my data between [0,1], and have a vector like [0, 0.001, 0.01, 0.1, 1], is that mean ...
0
votes
0answers
8 views

Input format of label images (ImageDataGenerator) for multi-class semantic image segmentation in keras

I wondered if you could help me (and hopefully others too) to understand how to use keras' ImageDataGenerator to load in my label_masks and zip them with the input_images for semantic segmentation. My ...
0
votes
0answers
8 views

How to get model predictions on all classes after applying Transfer Learning?

I have been following transfer learning with TFHub to implement transfer learning in my model for text classification. However, I do not understand how to get probabilities for all the classes (1000 ...
0
votes
1answer
22 views

CNN Architecture for Multiple Instance Learning

I have a binary classification problem where I have a bag of documents (image files) that I need to classify - the bag that is, not the individual document. However, a bag can have a different number ...