Questions tagged [neural-network]
Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.
4,362
questions
0
votes
0
answers
19
views
Why isn't the validation data loss close to the test data loss?
First I set aside about 15% of my data as test data. Then, I used tensorflow.keras to create a relatively simple neural net model. Then I set the model.fit() parameter validation_split=0.2, so 20% of ...
- 1
0
votes
0
answers
22
views
Is there a Tensorflow built-in function to create a matrix from a single-layered feedforward neural network without activation functions?
In Tensorflow, I implemented a simple single-layer feedforward neural network with N inputs and N outputs without activation functions and biases. Simply, it is just a N-by-N matrix. Question: is ...
- 1
0
votes
0
answers
4
views
Accuracy Drop in ViT with Patch Embedding: Investigating the Impact of Added Convolutional Layers
I'm currently working on incorporating a patch embedding layer into my Vision Transformer (ViT). I've defined this layer using four 2D convolutional and initialized it with a normal distribution. The ...
2
votes
1
answer
115
views
Why the test accuracy showing some odd behaviour in comparison to train accuracy?
I am currently training an ANN using Sequential(a class from Keras API within tensorflow), and I am optimizing the model's architecture and came across something I have not seen before.
The graph of ...
- 23
0
votes
0
answers
11
views
Packet data needed fot the model
I want to use this data set https://huggingface.co/datasets/rdpahalavan/network-packet-flow-header-payload for network attack classification
Here is packet structure from the README file.
Assuming I ...
- 101
0
votes
1
answer
20
views
Holding batch size constant, will a bigger dataset consume more GPU memory?
If you hold (mini) batch size constant (as well as everything else) but increase the number of examples (and therefore the number of training iterations), should you expect a (significant) increase in ...
- 101
0
votes
0
answers
77
views
Neural Net not able to learn simple analytical equation
I am currently making my first attempts with Pytorch. I am trying to solve a simple equation with a neural net. Analytically solved, the result of my neural net shall look like this:
$$
y = \frac{x_5}{...
- 1
1
vote
1
answer
32
views
activation=tf.keras.activations.relu vs activation='relu'
Both models are for binary classification problems
Model 1
...
0
votes
1
answer
26
views
Confusion with tensorflow's Sequential Dense Layers
I'm working on a regression probem using Tensorflow, and have created two models with slight differences in their first Dense layer.
The Models
...
0
votes
0
answers
31
views
How was the word2vec model trained?
Let's take the CBOW (continuous bag of words) model as the example.
Suppose that, there are $c$ context words, each of which is a one-hot encoding vector. So the total number of elements of input ...
- 46
0
votes
1
answer
62
views
Neural Network for binary classification not working
I have made a neural network that was working correctly as a multi-class classifier, but after changing the loss and the activation function, plus the output layer to just 1 neuron, it is not working ...
0
votes
0
answers
23
views
Should non-trainable functions be part of a nn model?
Some explanation for the somewhat obscure title:
I want to train a model which can produce images given some input data. However, actually I want the model to learn some abstract representation which ...
- 101
0
votes
1
answer
36
views
Reinforcement learning
I am working on a sentiment analysis project. I used BERT model for training but lack of data it gives huge overfitting. So after i moved LLM approach to do that. Using LLM finally i got good results....
0
votes
0
answers
9
views
Loss MAE when estimating the angle of rotation of an object in an image is stuck at about 90
I am dealing with the problem of estimating the angle of rotation of objects in images. The problem is that the network gets stuck when training at a loss level of about 90.
Below is the code for my ...
- 1
1
vote
0
answers
21
views
Determining the start and stop of vehicle movement by GPS
There is a large fleet, slightly more than a million vehicles, which is constantly growing. Each vehicle sends GPS coordinates to the server, as well as events (ignition is on, door is open, parking ...
0
votes
0
answers
15
views
How to choose the correct NN model if the metrics are different in training and test time?
I am trying to build an LSTM model which has a lot of Dropout and Batch Norm Layers. When I run model.fit, the accuracy comes out to around 0.7 on the training data....
0
votes
0
answers
14
views
How to choose the right typt of ANN architecture for a regression model
So, im working on a project where i am leveraging ai to get accurate price predictions in terms of houses and real estate properties. I would like to use an artificial neural network so now i have to ...
- 1
0
votes
0
answers
36
views
Avoid killing learnable parameters when transforming input into intervals
I'm trying to make a model using Pytorch which is training and transforming a set of coordinates, and then is downsampled using the model below. However, when I'm making the input coordinates into ...
- 1
0
votes
0
answers
20
views
Feeding more data to a neural network
I watched a video on Tesla's FSD where the drive was really smooth but required one intervention when the traffic light changed to green but the car wouldn't go because it looked like the light was ...
- 1
1
vote
0
answers
10
views
What is a proper activation function with simulated-annealing trainer for neural network?
I'm developing a gpu-accelerated simulated annealing based neural network trainer library. Currently its stuck on how to converge on "array sorting by neural network 3:10:20:10:3 topology".
...
1
vote
0
answers
28
views
ML paper reproducibility
How can I reproduce results in an ML paper if I don't have the identical resources to train the models as in the paper ? (in my case I only have a laptop spec NVidia gpu and in most of the papers I ...
- 11
0
votes
0
answers
18
views
3D Design file labelling and classification for manufacturing
I have ~1 million 3D design (.STP and/or .OBJ) files of various parts for medical devices, aerospace, automotive or defense systems. I'd like to label them based on appropriate manufacturing methods ...
- 1
0
votes
0
answers
8
views
Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?
I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
- 101
0
votes
0
answers
40
views
Custom Loss Function Returns Graph Execution Error: Can not squeeze dim[0], expected a dimension of 1, got 32
I have built a loss function which adds time and frequency weighted averages and variances to the MSE:
...
1
vote
1
answer
30
views
LSTM Training Interpretation
I have an LSTM and have the following chart showing training validation performance by epoch:
Could someone explain? How can my validation performance be better than my training performance in ...
- 143
0
votes
0
answers
13
views
RMSE of whole and part of test dataset
Can anybody help me to understand the behavior of metrics (RMSE namely) when testing model? I have NN with 1 hidden layer for regression task. RMSE equal 0.07 for external test dataset. But if I break ...
- 1
0
votes
0
answers
26
views
Using Embedding For Regularization
Is using embeddings for regularization a valid practice?
My reasoning for that is that encoding training/tests datasets into smaller vectors would allow a smaller network with fewer parameters and ...
0
votes
0
answers
45
views
Alternative to ELU and Leaky ReLU?
I was talking with a friend about different activation functions (we are still new to ML). One thing that I didn't like about ELU was the vanishing gradient, and about Leaky ReLU that it's not ...
- 1
0
votes
0
answers
28
views
Is it the right approach to select the model when it gives highest accuracy on validation dataset?
I am training the Densenet121 Model on an image dataset. I divided the dataset into 80% for training and 20% for testing.
Then I further divided the training data into 85% for training and 15% for ...
0
votes
0
answers
28
views
How to handle different sequence sizes during training and production use?
My training data consists of sequences that are mode of tokens.
The objective of my model is to predict score for each token in the input sequence. This score depends on the particular token itself, ...
- 101
1
vote
0
answers
17
views
Are "textbook backpropagation" still relevant?
The above backpropagation algorithm is taken from Shalev Shwartz and Ben-David's textbook: Understanding Machine Learning. This algorithm is described in the same way as the one in Mostafa's textbook, ...
- 119
0
votes
0
answers
12
views
L2 regularisation included in Validation Loss is counter intuitive?
I have been trying to tune hyperparameters for a neural network - I noticed the validation data loss for tensorflow in particular includes the L2 regularisation loss as a measure of the total loss.
...
- 101
0
votes
0
answers
13
views
eXplainable Artificial Intelligence (XAI). Need help building a XAI model to explain the results of an IDS classifier
I need some help building a XAI model with Keras to explain the results of an MLP working as an IDS. I have resarched about XAI but the only thing I find is small portions of code that just use ...
1
vote
0
answers
29
views
Convolutional networks: remove useless features?
I'm new to convolutional neural networks and have two related questions:
If all the filters would have the same weights initially, they would all detect the very same feature - so it would be useless ...
- 111
0
votes
0
answers
43
views
Can we show only results of some epochs in tqdm?
I am training a NN and use tqdm for showing the results. However, the bad thing is that it shows the results for every epoch. This is too many as I want to train NN for atleast 500 epochs.
Is there ...
- 57
0
votes
0
answers
7
views
Parametrizing a decreasing curve
I am trying to estimate a curve that by construction has to start from (0,0) and be decreasing. My current approach is to predict 20 numbers $d_i$ on [0, 10) as the differences between values on the ...
- 101
1
vote
2
answers
68
views
I build my first neural network! What's next?
For an assignment we are given a dataset and we need to build a neural network to make new predictions.
Personal milestone; I built my first neural network!
I followed the following steps on my own ...
- 111
0
votes
2
answers
92
views
How to improve accuracy on a single class out of 3 classes in model
I am training a classification model with 3 classes using a deep neural network.
The classes have been resampled and balanced.
I have around 600000 samples... equally distributed.
The dataset is also ...
- 125
0
votes
3
answers
73
views
Reduce false positives having imbalanced data
I'm using a DNN-48 having the following scenario:
Features: 8 (48 at the end because I generate conditional sequences of 6 elements each)
Classes: Y=0 (90%), Y=1 (10%)
Precision and recall are good ...
- 3
0
votes
1
answer
48
views
This simple python Feed forward Neural Network isn't learning. What am I doing wrong?
The backpropagation procedure is taken from the approach outlined in here.
Here is the code, commented:
...
- 103
0
votes
0
answers
31
views
Training loss is much higher than validation loss
I am trying to train a neural network with 2 hidden layers to perform a multi class classification of 3 different classes. There is a huge imbalance to the classes, with the distribution being around ...
0
votes
0
answers
55
views
Why I am getting error in dataloader in defining a NN?
I am trying to write a NN. However I am getting error.
Here is my Code:
...
- 57
0
votes
0
answers
7
views
Dynamic dosing recipe for accurate pH
I want to have a script for adjusting dosage of ingredients in each batch dynamically. Assuming that the requirement is to have a specific value of pH from each batch but with the variation of raw ...
0
votes
0
answers
11
views
Questions regarding backward propagation
I am reading article regarding backward propagation https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ . Lets say if I follow the example in the article but using only 3 node, ...
- 1
0
votes
1
answer
29
views
Appropriate input size for nn.Embedding
I’m quite new to using Pytorch and deep learning. What size of unique categories of a categorical variable is appropriate for applying the nn.Embedding ideally (best practices)? for example, if a ...
1
vote
0
answers
21
views
Why do I keep on getting ResourceExhaustedError while training on video data using CONV3D on tensorflow?
I'm encountering a memory allocation problem while training a deep learning model on my computer, which has a Core i9 10th Gen CPU, 64 GB of RAM, and an NVIDIA GTX 1660 Super with 6GB of VRAM. Despite ...
- 11
1
vote
1
answer
51
views
Why my validation loss and accuracy decays over epochs?
Im trying to build 2 simple networks with cleaned dataset for tweets sentiment classification(0/1):
one with all dense layers(binary bag of words)
another with RNN layer(embedding layer).
But it both ...
- 35
0
votes
1
answer
34
views
Tensorflow diagram for attention mechanism
I was reading the tutorial from tensorflow on the transformer model, however, when they explain the transformer model, they display such a picture :
which I don't understand. What do the ingoing ...
- 101
0
votes
0
answers
26
views
Multi input model tensorflow with fixed data as input
I am tring to implement the following architecture.
alpha and beta are fixed matrices, they are matrices I want to input on every forward pass. Meaning for every batch they should be the same My ...
0
votes
0
answers
15
views
Making a NN closer to a linear regression?
It is possible to 'initialise' a gradient boosted model with a simpler model, such as linear regression, by manually setting the initial score. This seems to help reduce the discrepencies between the ...
- 2,567