Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
1
vote
1answer
41 views

Vanishing gradient problem even after existence of ReLu function?

Let's say I have a deep neural network with 50 hidden layers and at each neuron of hidden layer the ReLu activation function is used. My question is Is it possible for vanishing gradient problem to ...
0
votes
1answer
43 views

Early stopping with class weights / sample weights

I'm performing a classification of imbalanced multiclass data using a Neural Network in the TensorFlow framework. Therefore, I'm applying class weights. I would like to apply early stopping to reduce ...
2
votes
2answers
81 views

KL-divergence to compare ML models

Let us say we have to neural network architectures, A and B and we train $x$ times each of them. Based on the $x$ retrainings, we can calculate $x$ prediction errors for each model, and plot its ...
0
votes
0answers
35 views

text classification - does number of features matters?

I'm working on a multi-class text classification project that aims to assign a "new bug" to his "final group assignee" To do that I was able to extract ~17000 samples and divided ...
0
votes
1answer
18 views

Question regarding training data in word2vec - skip-gram

I have a very simple question regarding the training data in word2vec. In the skip-gram implementation, the training data (if I understand it correctly) is generated as pairs of words like it's shown ...
0
votes
0answers
21 views

Don't understand Channels in Covolutional Layers [duplicate]

I'm struggling to understand the concept of 'Channels'. What does a channel mean in the context of an image. I understand that a grey scale image only has 1 channel, and a RGB has 3, but then I see ...
0
votes
0answers
10 views

What is the use of the ID field in the source code?

Building a One Hot Encoding Layer with TensorFlow One-Hot Encoder Check out the following source code: ...
1
vote
1answer
45 views

Conditional variational autoencoder: Feeding labeled MNIST to encoder with Keras

I am looking for a code implementation of a CVAE using MNIST in Keras. I found this Youtube video: https://youtu.be/8wrLjnQ7EWQ that does VAE, but I am not sure how do I convert this and make encoder ...
0
votes
0answers
23 views

Neural Net gradient descend

I was planning on making my own neural network library in C++ and was going through other's code to make sure I am on right track. Below is a sample code that I am trying to learn from. Everything in ...
0
votes
0answers
100 views

global average pooling in PyTorch: torch.nn.AvgPool1d vs torch.mean

To implement global average pooling in a PyTorch neural network model, which one is better and why: to use torch.nn.AvgPool1d() and set the ...
1
vote
1answer
39 views

Training neural network to emulate a hash function

A hash function takes an input, performs a set of complex operations and then produces an output. For my purposes the output from the function will always be the same for any given input. I remember ...
1
vote
0answers
18 views

how to calculate parameters of an RNN using backpropagation

I'm trying to find out the two binary inputs are identical or not using RNN. my architecture is like this: I have the following functions: Where vT is the transpose of vector v and the activation ...
0
votes
0answers
20 views

How do convolutional layers in a CNN feed forward when there is multiple input feature maps?

I've been trying to recreate LeNet 1(LeNet 1 architecture is pictured in the top diagram) in python using NumPy. I am unsure of how the forward pass works when there is multiple Input feature maps in ...
0
votes
0answers
6 views

Problem in convergence of hebbian learning approach for Fuzzy Cognitive Map

I was trying to learn Fuzzy Cognitive Map by Active Hebbian Learning approach from here. What I have understand is that the model learns iteratively, at each step a new concept values enters and tune ...
0
votes
0answers
50 views

Character Level Embedding in Sentence Classification

I'm working on an NLP task that requires the use of character level embeddings. By using tokenizer library I realized that it tokenizes such as lower integer meant the most frequent character. Is ...
1
vote
0answers
24 views

Adding noise after LSTM layer

I am building a Natural Language Inference neural network model that learns to identify if one sentence (hypothesis) follows from another sentence (premise). So the input to my network is 2 sentences, ...
0
votes
0answers
10 views

How to deal with catastrophic forgetting?

I have my own implementation of ppo, which I've been trying to train for days on BreakoutNoFrameskip-v4 after totally failing to get a2c past a mean reward of 10 ...
1
vote
0answers
14 views

Account for imbalanced data in a Neural Network using prior distribution

I have a dataset with 4 classes, say their distribution in the training-set is $P_{prior}(C1) = 60\% $ $P_{prior}(C2) = 25\% $ $P_{prior}(C3) = 10\% $ $P_{prior}(C4) = 5\% $ After training a Neural ...
0
votes
0answers
19 views

CNN heatmaps substantially different for different input images

I have a convolution neural network for regression, where medical scans of many people are trained to predict some continuous variable (body related phenotype). I get reasonable performance (R2 ~ 0.9)....
1
vote
1answer
41 views

modeling time series data with large number of variables

I want to model time series data of 52 dependent variable using neural networks in order to forecast these series in future . I have tried some architectures of LSTM and CNN (conv1D) models but my ...
1
vote
1answer
44 views

sklearn MinMaxScaler: Inverse does not equal original

I am using MinMaxScaler on a large dataset (2201887, 3) to normalize features. Inversed values does not match originals. I tested with the target column, first (a), I applied the scaler on 10 values, ...
1
vote
0answers
21 views

Why won't my TFJS model's accuracy exceed .508 despite loss decreasing, and the fact that it worked for a different dataset (Iris dataset)?

This post is aptly titled: my stock prediction model's accuracy just won't go past 0.5088282227516174 despite loss decreasing. I have tried so many different things, such as: Increasing batch size ...
1
vote
0answers
39 views

LSTM Many to one with multiple time steps for time series (multi class classification)

I want to do a time series multi-class classification for fault detection and diagnosis with time-series sensor data set which contains a sequence of 50 records of normal data and sequences of another ...
2
votes
1answer
41 views

Why first fully connected layer requires flattening in cnn?

One can read everywhere on internet or in books that in convoluted neural networks, between convolution layers and the first fully connected layer, you should flatten your data. I managed to ...
1
vote
0answers
10 views

Compare rate of change for multiple object/weights

For a Neural Network, the weight update equation is: However, there are millions of such weights W_i. If I am interested in capturing how much each weight/connection W_i is changing as compared to ...
0
votes
0answers
22 views

StarGAN How to test discriminator

I'm running the following code: https://github.com/taki0112/StarGAN-Tensorflow I have my model pretrained. After the training I want to run the discriminator function to check its accuracy. Assume ...
1
vote
2answers
28 views

Can I use labelled data in unsupervised learning algorithms like neural network?

I am working on a transaction dataset that consists of some labeled features like gender, product categories, membership types, and so on. There are also some numeric data like the amount of ...
0
votes
0answers
15 views

Proof of Correctness of Perceptron Training Rule

The Perceptron Training Rule is basically applying Stochastic Gradient Descent for finding the coefficients of a hyperplane (which works as a Decision Boundary) for doing binary classification of data ...
1
vote
0answers
31 views

How to train a deep neural network to return the input as it is?

The task is to train a neural network to return the input as it is, like X -> X or Y -> Y. The network should contain at ...
0
votes
1answer
67 views

Loss is Nan even with clipvalue set and Adam optimizer

I'm currently doing this task from kaggle. I've normalized my data with the minmax scaler and fixed the dummie variable trap by removing one column to every dummie variable I created. Here is the ...
0
votes
0answers
14 views

How can i improve my Bidirectional LSTM timeseries forecasting

I am trying to forecast a timeseries and I am using LSTM for it. But the forecast outside train data is pretty bad. I tried adding layers, changing epochs but couldn't improve. The forecast is ...
2
votes
1answer
52 views

Implementing computational graph and autograd for tensor and matrix

I am trying to implement a very simple deep learning framework like PyTorch in order to get a better understanding of computational graphs and automatic differentiation. I implemented an automatic ...
1
vote
1answer
31 views

Is there comprehensive list of activation functions and their applications for a Neural Network?

I am aware of common activation functions like sigmoid, tanh,ReLu, Leaky ReLu. Even heard about a function called Swish. Now is there any detailed information on other activations functions and some ...
3
votes
1answer
275 views

Why can a neural network solve XOR if it is not a continuous function?

My understanding is that a typical neural network without any fancy activation functions can only solve problems that can be modelled as a continuous function. If this is the case, why can a standard ...
2
votes
0answers
26 views

Regress values inside the bounding boxes to predict a value in Object Detection

I am currently working on an object detection task. I have a dataset of grayscale and depth images. The annotation format is $x_1, y_1, x_2, y_2, class, depth$. I calculated this depth (of each ...
2
votes
2answers
39 views

Should I train the "Unknown" class separately from the other classes

I have a CNN model that classifies 10 classes of audio spectrograms. However, since I work with the open set of data, I need to classify the unknown audio data as an "Unknown" class. The ...
2
votes
0answers
11 views

Addressing class imbalance: coefficient in loss vs threshold on confidence

Let's say I am training a neural network for a binary classification task, with classes A and B, which is trained using gradient descent with a cross-entropy loss function. The dataset is imbalanced, ...
0
votes
0answers
23 views

Predicting a distribution (instead of a point estimate) with supervised machine learning

I'm new to machine learning, but it seems that supervised learning algorithms that aren't also considered statistical models (random forests, neural networks, etc...) are focused on predicting a point ...
2
votes
1answer
65 views

Using Subsequent Mask in Transformer Leads to NaN Outputs

I am trying to implement an autoregressive transformer model similar to the paper attention is all you need. From what I have understood, in order to replicate the architecture fully, I need to give ...
1
vote
1answer
23 views

Studying and choosing between different neural network structures

I would like to develop a model that uses convolutional neural networks for image classification. From the many different network structures described in papers and articles online, I would like to ...
0
votes
1answer
23 views

Memorization in deep neural networks, random vs. properly labelled datasets

From about 19:20 in the video here: https://www.youtube.com/watch?v=IHZwWFHWa-w it shows the difference in value of the cost function for randomly labelled data vs. properly labelled data. What do ...
0
votes
1answer
24 views

How to represent a "switch"-like behavior in a neural network?

I have three input variables $x_1$, $x_2$ and $d$, where $x_1$ and $x_2$ are numerical variables and $d$ is a dummy variable that takes the value of 1 or 2. How to represent the part of a neural ...
0
votes
0answers
21 views

Any recomended Neural Network structure to classify similar images?

I have a number of very similar images, and I want to classify them into two groups. I used Resnet50 before, but the network does not seem to be learning at all. As the training progresses, the ...
1
vote
0answers
43 views

How does Keras optimization for a network with multiple outputs

I currently have a neural network that takes in 3 numbers as inputs and outputs 3 numbers. I've attached a picture of the network below and my code is accessible through the following link: [Google ...
2
votes
1answer
54 views

MOOC - for causal analysis - no statistics background [closed]

Am a software guy with no background in causal inference. While I am now familiar with prediction techniques due to the plethora of courses available online, I would like to seek recommendations from ...
2
votes
1answer
15 views

Effect of batch size on dense networks?

I have been doing DS for a couple of years now and have returned to "tinkering" a bit more with toy data sets and overall just honing my skills a bit. I was recently playing with a very ...
1
vote
0answers
48 views

Implementing cosine embedding loss with labels 0 and 1

My dataset has two labels, 0 and 1, 1 meaning high similarity and 0 meaning high dissimilarity. Two output vectors from the two-tower model are compared using dot product (with normalization) and ...
1
vote
0answers
16 views

Using Z-test score to evaluate model performance

I think I know the answer to this question but I am looking for a sanity check here: Is it appropriate to use z-test scores in order to evaluate the performance of my model? I have a binary model that ...
0
votes
0answers
182 views

How do I solve a "TypeError: __array__() takes 1 positional argument but 2 were given" Keras error?

I am trying to build a multi-input CNN using Keras/Tensorflow. I have 5000 'smile' training inputs which are 1D arrays (shape = (100,)). These inputs have a maximum length of 100. I have 5000 'protein'...
2
votes
1answer
27 views

What is fully connected layer additive bias?

I'm going to use PyTorch specifically but I suspect my question applies to deep learning & CNNs in general therefore I choose to post it here. Starting at this point in this video and subsequently:...

1 2
3
4 5
78