Questions tagged [neural-network]
Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.
4,349
questions
0
votes
0
answers
9
views
L2 regularisation included in Validation Loss is counter intuitive?
I have been trying to tune hyperparameters for a neural network - I noticed the validation data loss for tensorflow in particular includes the L2 regularisation loss as a measure of the total loss.
...
- 101
0
votes
0
answers
5
views
eXplainable Artificial Intelligence (XAI). Need help building a XAI model to explain the results of an IDS classifier
I need some help building a XAI model with Keras to explain the results of an MLP working as an IDS. I have resarched about XAI but the only thing I find is small portions of code that just use ...
1
vote
0
answers
17
views
Convolutional networks: remove useless features?
I'm new to convolutional neural networks and have two related questions:
If all the filters would have the same weights initially, they would all detect the very same feature - so it would be useless ...
- 111
0
votes
0
answers
19
views
Can we show only results of some epochs in tqdm?
I am training a NN and use tqdm for showing the results. However, the bad thing is that it shows the results for every epoch. This is too many as I want to train NN for atleast 500 epochs.
Is there ...
- 1
0
votes
0
answers
6
views
Parametrizing a decreasing curve
I am trying to estimate a curve that by construction has to start from (0,0) and be decreasing. My current approach is to predict 20 numbers $d_i$ on [0, 10) as the differences between values on the ...
- 101
1
vote
2
answers
54
views
I build my first neural network! What's next?
For an assignment we are given a dataset and we need to build a neural network to make new predictions.
Personal milestone; I built my first neural network!
I followed the following steps on my own ...
- 111
0
votes
2
answers
52
views
How to improve accuracy on a single class out of 3 classes in model
I am training a classification model with 3 classes using a deep neural network.
The classes have been resampled and balanced.
I have around 600000 samples... equally distributed.
The dataset is also ...
- 125
0
votes
3
answers
56
views
Reduce false positives having imbalanced data
I'm using a DNN-48 having the following scenario:
Features: 8 (48 at the end because I generate conditional sequences of 6 elements each)
Classes: Y=0 (90%), Y=1 (10%)
Precision and recall are good ...
- 3
0
votes
1
answer
38
views
This simple python Feed forward Neural Network isn't learning. What am I doing wrong?
The backpropagation procedure is taken from the approach outlined in here.
Here is the code, commented:
...
- 103
0
votes
0
answers
18
views
Training loss is much higher than validation loss
I am trying to train a neural network with 2 hidden layers to perform a multi class classification of 3 different classes. There is a huge imbalance to the classes, with the distribution being around ...
0
votes
0
answers
27
views
Why I am getting error in dataloader in defining a NN?
I am trying to write a NN. However I am getting error.
Here is my Code:
...
- 1
0
votes
0
answers
7
views
Dynamic dosing recipe for accurate pH
I want to have a script for adjusting dosage of ingredients in each batch dynamically. Assuming that the requirement is to have a specific value of pH from each batch but with the variation of raw ...
0
votes
0
answers
10
views
Questions regarding backward propagation
I am reading article regarding backward propagation https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ . Lets say if I follow the example in the article but using only 3 node, ...
- 1
0
votes
1
answer
18
views
Appropriate input size for nn.Embedding
I’m quite new to using Pytorch and deep learning. What size of unique categories of a categorical variable is appropriate for applying the nn.Embedding ideally (best practices)? for example, if a ...
1
vote
0
answers
11
views
Why do I keep on getting ResourceExhaustedError while training on video data using CONV3D on tensorflow?
I'm encountering a memory allocation problem while training a deep learning model on my computer, which has a Core i9 10th Gen CPU, 64 GB of RAM, and an NVIDIA GTX 1660 Super with 6GB of VRAM. Despite ...
- 11
1
vote
1
answer
48
views
Why my validation loss and accuracy decays over epochs?
Im trying to build 2 simple networks with cleaned dataset for tweets sentiment classification(0/1):
one with all dense layers(binary bag of words)
another with RNN layer(embedding layer).
But it both ...
- 35
0
votes
1
answer
26
views
Tensorflow diagram for attention mechanism
I was reading the tutorial from tensorflow on the transformer model, however, when they explain the transformer model, they display such a picture :
which I don't understand. What do the ingoing ...
- 101
0
votes
0
answers
16
views
Multi input model tensorflow with fixed data as input
I am tring to implement the following architecture.
alpha and beta are fixed matrices, they are matrices I want to input on every forward pass. Meaning for every batch they should be the same My ...
0
votes
0
answers
15
views
Making a NN closer to a linear regression?
It is possible to 'initialise' a gradient boosted model with a simpler model, such as linear regression, by manually setting the initial score. This seems to help reduce the discrepencies between the ...
- 2,523
0
votes
0
answers
17
views
Training a two-layer neural network for multi-label data (binary bit array of dim 50)
This is my problem setup.
Train Input size (6300x300) These are standard BERT embeddings, so floating point numbers, mostly negatives.
Train Output size (6300x50) These are binary bit arrays like [0, ...
- 101
0
votes
1
answer
30
views
What Model to Choose for a NN with a Very Wide Output Layer?
The input of my neural network consists of 20 features, whereas the output consists of 20,000 of them (predicting a "quantum classical shadow" based on a few parameters: the rotation angle ...
- 11
0
votes
1
answer
35
views
How is the backward propagation is done in pytroch? When to use torch.no_grad, also when and where is the gradinte calcuated?
I have this training loop in pytorch. the loss_fn = nn.CrossEntropyLoss() and optim = torch.optim.Adam(net.parameters(), lr=lr)
<...
0
votes
0
answers
14
views
How can BERT/Transformer models accept input batches of different sizes?
I understand that all inputs in a batch need to be of the same size. However, it seems BERT/Transformers models can accept batches with different sizes as input.
How is that possible? I thought we ...
- 1
0
votes
1
answer
23
views
Custom Loss Function in Tensorflow for UNet
I am working on a Segmentation task, where I planned to use U-Net
for the input_image of shape (224,224,3), the output should be the mask image of shape ...
- 21
2
votes
1
answer
47
views
What is the benefit of the exponential function inside softmax?
I know that softmax is:
$$ softmax(x) = \frac{e^{x_i}}{\sum_j^n e^{x_j}}$$
This is an $\mathbb{R}^n \implies \mathbb{R}^n$ function, and the elements of the output add up to 1. I understand that the ...
- 123
0
votes
0
answers
11
views
Mapping two unrelated representation spaces with different distributions, roughly preserving similarities
I've got two embedding / representation spaces which are completely unrelated to begin with, yet I wish to find a (=any) mapping between them.
Space A are feature histograms with a somewhat normal ...
- 1
0
votes
1
answer
22
views
different range of target values in neural network
I am working on a neural network regression code. The dataset includes 14 features in the range value between -1 and 1. while the target variable is changing among (0.000759) to (1100).
The target ...
- 1
0
votes
0
answers
9
views
Multi-input NetGraph configurations with Pytorch
I'm working on an advanced reinforcement learning problem with PyTorch and the environment we'd like to present to the agent consists of 3 inputs mapping to a 245-...
- 121
1
vote
1
answer
38
views
Finding parameter combinations for zero gradients in an artificial neural network
Consider the following network:
There are two weights, say $w_1$ and $w_2$, and two biases, $b_1$ and $b_2$. The hidden layer has a ReLU activation function $g^{(1)}$ and the output layer has a ...
- 147
0
votes
2
answers
48
views
what is a random prediction for class imbalanced data? How can I check if my model is predicting randomly?
Say if you have a balanced dataset, with two classes, if the classification model that we’re training doesn’t learn anything ( suppose the data is random ), the model’s output would be 50% first class ...
1
vote
2
answers
77
views
Gradient Descent: Is the magnitude in Gradient Vectors arbitrary?
I am only just getting familiar with gradient descent through learning logistic regression. I understand the directional component in the gradient vectors is correct information derived from the slope ...
- 11
0
votes
1
answer
21
views
How does a VQ-VAE produce new images?
I'm implementing a VQ-VAE for a LDM for biological time series data. I trained the VQ-VAE, and reconstructions works somewhat reasonable, but I have an understanding problem with how a VQ-VAE works.
...
- 11
2
votes
3
answers
101
views
What is the most optimal machine learning model/algorithm to create a hangman solver?
Want to create a hangman solver, So what is the best ml algorithm (lstm,reinforcement learning, or etc) to use?
Do suggest any other optimal technique if you know?
- 21
0
votes
0
answers
40
views
Train neural network to predict multiple distributions
I aim to train a neural network to predict 2 distributions (10 quantiles, i.e. deciles) at 5 time points. So my y is of shape:
...
- 30
0
votes
0
answers
19
views
Feed two matrices into an mlp neural network
So Im trying to train a recommender system (w/ DQN) using two sets of data , first is a 2D array size $N\times N$ where the diagonal is the current content (= state) and the rest row is the ...
0
votes
0
answers
23
views
Are there other "interactive" non-linear neural network layers besides self-attention layer?
In the self-attention layer
$$
\operatorname{Attention}(Q, K, V)=\operatorname{softmax}\left(\frac{Q K^T}{\sqrt{d_k}}\right) V
$$
$Q$, $K$ and $V$ are all linear with respect to embedding vectors $x$, ...
- 215
0
votes
0
answers
26
views
Gradients of lower layers of a CNN when gradient of an upper layer is 0?
Say we have a convolutional neural network with an input layer, 3 convolutional layers and an output layer.
Say the gradients with respect to the weights and biases of the third convolutional layer ...
- 147
2
votes
1
answer
252
views
Gradients of lower layers of NN when gradient of an upper layer is 0?
Say we have a neural network with an input layer, a hidden layer and an output layer.
Say the gradients with respect to the weights and biases of the output layer are all 0.
Then, by backpropagation ...
- 147
2
votes
2
answers
101
views
What does it mean order of input sequence does not matter for transformer self-attention head?
The need for positional encoding in transformer models is justified by permutation invariance of self-attention heads, because, without it, transformer wouldn't have any mechanism to take into account ...
- 215
0
votes
1
answer
25
views
Set a numerical range restriction on the output of a model
I am trying to train two models together. Both are regression models.
The first model would output a prediction, which is fed into the second model.
The second model is pre-trained to make good ...
0
votes
0
answers
12
views
Using neural networks to learn total distributions from linearly transformed samples
I am trying to explore the use of neural network based models to replace a cumbersome challenge with a faster more approximate method. The problem involves learning a distribution termed the "...
- 121
0
votes
0
answers
42
views
Backpropagation and Gradient Descent: Questions on math behind it
I watched this video which goes over backpropagation calculus and read the Wikipedia page on it. This is my understanding of the equations for the algorithm. I have questions regarding the equations ...
- 101
0
votes
0
answers
55
views
Exploding loss in unstable model
I am training a classifier (based on transformer encoders), on top of some complex data. My data is extremely imbalanced (although I do undersample the higher concentration class somewhat) and rather ...
- 101
1
vote
1
answer
26
views
ELMO embeddings
Could somebody tell me how does elmo work?
Is it good for phrase embedding too?
I m looking for phrase embeddings.
Thank You in advance.
1
vote
1
answer
25
views
Patterns binary classification - model doesn't overfit
I am working on a very basic binary classification problem. For each set of four float numbers $(x,y,z,w)$, I want to check if they fall or not into one category.
I have written a model in Keras with ...
- 121
0
votes
0
answers
20
views
Question about Elements of Statistical Learning - Section 11.7
This might be a very easy question but somehow I am stuck. In this chapter, Net-3 has 16x16 input layer and 8x8 first hidden layer. It is using local connectivity with 3x3 input patch. The receptive ...
1
vote
0
answers
23
views
What machine learning technique can help generate spectrum line profiles?
I'm trying to work with Calcium-K line profiles from the Sun.
Image for reference. Please ignore the labels on the image and note that my profiles are not in image format (more info below).
I have ...
0
votes
0
answers
19
views
consultation ,Calculating Weight Adjustments for hidden-Layer Neurons
In the calculations for the given network, I understood the Forward-Pass Calculation calculations, but in the Backward-Pass Calculation section; When calculating Calculating Weight Adjustments for ...
- 1
0
votes
1
answer
37
views
How to train the AI to recognize soldiers' allegiance by armband?
If I hypothetically want to train an AI to recognize enemy soldiers by the color of their armband (for example green armband), should I feed the AI with only green armband soldiers or should I also ...
- 101
0
votes
0
answers
40
views
The cost function gets stuck at 120 epochs
I did a neural network in c++ to recognize handwritten digits using the MNIST dataset without any neural network pre-existing libraries. My network has 784 inputs neuron (the pixel of the image), 100 ...
- 1