Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
0 votes
0 answers
19 views

Why isn't the validation data loss close to the test data loss?

First I set aside about 15% of my data as test data. Then, I used tensorflow.keras to create a relatively simple neural net model. Then I set the model.fit() parameter validation_split=0.2, so 20% of ...
Alex's user avatar
  • 1
0 votes
0 answers
22 views

Is there a Tensorflow built-in function to create a matrix from a single-layered feedforward neural network without activation functions?

In Tensorflow, I implemented a simple single-layer feedforward neural network with N inputs and N outputs without activation functions and biases. Simply, it is just a N-by-N matrix. Question: is ...
H.C.'s user avatar
  • 1
0 votes
0 answers
4 views

Accuracy Drop in ViT with Patch Embedding: Investigating the Impact of Added Convolutional Layers

I'm currently working on incorporating a patch embedding layer into my Vision Transformer (ViT). I've defined this layer using four 2D convolutional and initialized it with a normal distribution. The ...
External_Happy_77's user avatar
2 votes
1 answer
115 views

Why the test accuracy showing some odd behaviour in comparison to train accuracy?

I am currently training an ANN using Sequential(a class from Keras API within tensorflow), and I am optimizing the model's architecture and came across something I have not seen before. The graph of ...
Aach_copro's user avatar
0 votes
0 answers
11 views

Packet data needed fot the model

I want to use this data set https://huggingface.co/datasets/rdpahalavan/network-packet-flow-header-payload for network attack classification Here is packet structure from the README file. Assuming I ...
Roma's user avatar
  • 101
0 votes
1 answer
20 views

Holding batch size constant, will a bigger dataset consume more GPU memory?

If you hold (mini) batch size constant (as well as everything else) but increase the number of examples (and therefore the number of training iterations), should you expect a (significant) increase in ...
ubadub's user avatar
  • 101
0 votes
0 answers
77 views

Neural Net not able to learn simple analytical equation

I am currently making my first attempts with Pytorch. I am trying to solve a simple equation with a neural net. Analytically solved, the result of my neural net shall look like this: $$ y = \frac{x_5}{...
Ripleys's user avatar
1 vote
1 answer
32 views

activation=tf.keras.activations.relu vs activation='relu'

Both models are for binary classification problems Model 1 ...
Justin Jonany's user avatar
0 votes
1 answer
26 views

Confusion with tensorflow's Sequential Dense Layers

I'm working on a regression probem using Tensorflow, and have created two models with slight differences in their first Dense layer. The Models ...
Justin Jonany's user avatar
0 votes
0 answers
31 views

How was the word2vec model trained?

Let's take the CBOW (continuous bag of words) model as the example. Suppose that, there are $c$ context words, each of which is a one-hot encoding vector. So the total number of elements of input ...
J. Doe's user avatar
  • 46
0 votes
1 answer
62 views

Neural Network for binary classification not working

I have made a neural network that was working correctly as a multi-class classifier, but after changing the loss and the activation function, plus the output layer to just 1 neuron, it is not working ...
alex martinez's user avatar
0 votes
0 answers
23 views

Should non-trainable functions be part of a nn model?

Some explanation for the somewhat obscure title: I want to train a model which can produce images given some input data. However, actually I want the model to learn some abstract representation which ...
Roland Deschain's user avatar
0 votes
1 answer
36 views

Reinforcement learning

I am working on a sentiment analysis project. I used BERT model for training but lack of data it gives huge overfitting. So after i moved LLM approach to do that. Using LLM finally i got good results....
Sandun Tharaka's user avatar
0 votes
0 answers
9 views

Loss MAE when estimating the angle of rotation of an object in an image is stuck at about 90

I am dealing with the problem of estimating the angle of rotation of objects in images. The problem is that the network gets stuck when training at a loss level of about 90. Below is the code for my ...
DamianSz's user avatar
1 vote
0 answers
21 views

Determining the start and stop of vehicle movement by GPS

There is a large fleet, slightly more than a million vehicles, which is constantly growing. Each vehicle sends GPS coordinates to the server, as well as events (ignition is on, door is open, parking ...
Abraham Lincoln's user avatar
0 votes
0 answers
15 views

How to choose the correct NN model if the metrics are different in training and test time?

I am trying to build an LSTM model which has a lot of Dropout and Batch Norm Layers. When I run model.fit, the accuracy comes out to around 0.7 on the training data....
Jeffrey Davidson's user avatar
0 votes
0 answers
14 views

How to choose the right typt of ANN architecture for a regression model

So, im working on a project where i am leveraging ai to get accurate price predictions in terms of houses and real estate properties. I would like to use an artificial neural network so now i have to ...
maizen's user avatar
  • 1
0 votes
0 answers
36 views

Avoid killing learnable parameters when transforming input into intervals

I'm trying to make a model using Pytorch which is training and transforming a set of coordinates, and then is downsampled using the model below. However, when I'm making the input coordinates into ...
An alien's user avatar
0 votes
0 answers
20 views

Feeding more data to a neural network

I watched a video on Tesla's FSD where the drive was really smooth but required one intervention when the traffic light changed to green but the car wouldn't go because it looked like the light was ...
Noale's user avatar
  • 1
1 vote
0 answers
10 views

What is a proper activation function with simulated-annealing trainer for neural network?

I'm developing a gpu-accelerated simulated annealing based neural network trainer library. Currently its stuck on how to converge on "array sorting by neural network 3:10:20:10:3 topology". ...
huseyin tugrul buyukisik's user avatar
1 vote
0 answers
28 views

ML paper reproducibility

How can I reproduce results in an ML paper if I don't have the identical resources to train the models as in the paper ? (in my case I only have a laptop spec NVidia gpu and in most of the papers I ...
okm02's user avatar
  • 11
0 votes
0 answers
18 views

3D Design file labelling and classification for manufacturing

I have ~1 million 3D design (.STP and/or .OBJ) files of various parts for medical devices, aerospace, automotive or defense systems. I'd like to label them based on appropriate manufacturing methods ...
rootcage's user avatar
0 votes
0 answers
8 views

Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?

I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
BigMistake's user avatar
0 votes
0 answers
40 views

Custom Loss Function Returns Graph Execution Error: Can not squeeze dim[0], expected a dimension of 1, got 32

I have built a loss function which adds time and frequency weighted averages and variances to the MSE: ...
Harry Chittenden's user avatar
1 vote
1 answer
30 views

LSTM Training Interpretation

I have an LSTM and have the following chart showing training validation performance by epoch: Could someone explain? How can my validation performance be better than my training performance in ...
Windstorm1981's user avatar
0 votes
0 answers
13 views

RMSE of whole and part of test dataset

Can anybody help me to understand the behavior of metrics (RMSE namely) when testing model? I have NN with 1 hidden layer for regression task. RMSE equal 0.07 for external test dataset. But if I break ...
CUB_REV's user avatar
0 votes
0 answers
26 views

Using Embedding For Regularization

Is using embeddings for regularization a valid practice? My reasoning for that is that encoding training/tests datasets into smaller vectors would allow a smaller network with fewer parameters and ...
Adenilson Arcanjo's user avatar
0 votes
0 answers
45 views

Alternative to ELU and Leaky ReLU?

I was talking with a friend about different activation functions (we are still new to ML). One thing that I didn't like about ELU was the vanishing gradient, and about Leaky ReLU that it's not ...
Nasa's user avatar
  • 1
0 votes
0 answers
28 views

Is it the right approach to select the model when it gives highest accuracy on validation dataset?

I am training the Densenet121 Model on an image dataset. I divided the dataset into 80% for training and 20% for testing. Then I further divided the training data into 85% for training and 15% for ...
Dawood Ahmad's user avatar
0 votes
0 answers
28 views

How to handle different sequence sizes during training and production use?

My training data consists of sequences that are mode of tokens. The objective of my model is to predict score for each token in the input sequence. This score depends on the particular token itself, ...
Druudik's user avatar
  • 101
1 vote
0 answers
17 views

Are "textbook backpropagation" still relevant?

The above backpropagation algorithm is taken from Shalev Shwartz and Ben-David's textbook: Understanding Machine Learning. This algorithm is described in the same way as the one in Mostafa's textbook, ...
Fraïssé's user avatar
  • 119
0 votes
0 answers
12 views

L2 regularisation included in Validation Loss is counter intuitive?

I have been trying to tune hyperparameters for a neural network - I noticed the validation data loss for tensorflow in particular includes the L2 regularisation loss as a measure of the total loss. ...
Governor's user avatar
  • 101
0 votes
0 answers
13 views

eXplainable Artificial Intelligence (XAI). Need help building a XAI model to explain the results of an IDS classifier

I need some help building a XAI model with Keras to explain the results of an MLP working as an IDS. I have resarched about XAI but the only thing I find is small portions of code that just use ...
alex martinez's user avatar
1 vote
0 answers
29 views

Convolutional networks: remove useless features?

I'm new to convolutional neural networks and have two related questions: If all the filters would have the same weights initially, they would all detect the very same feature - so it would be useless ...
D.R.'s user avatar
  • 111
0 votes
0 answers
43 views

Can we show only results of some epochs in tqdm?

I am training a NN and use tqdm for showing the results. However, the bad thing is that it shows the results for every epoch. This is too many as I want to train NN for atleast 500 epochs. Is there ...
Ali.A's user avatar
  • 57
0 votes
0 answers
7 views

Parametrizing a decreasing curve

I am trying to estimate a curve that by construction has to start from (0,0) and be decreasing. My current approach is to predict 20 numbers $d_i$ on [0, 10) as the differences between values on the ...
Mark C.'s user avatar
  • 101
1 vote
2 answers
68 views

I build my first neural network! What's next?

For an assignment we are given a dataset and we need to build a neural network to make new predictions. Personal milestone; I built my first neural network! I followed the following steps on my own ...
Tim's user avatar
  • 111
0 votes
2 answers
92 views

How to improve accuracy on a single class out of 3 classes in model

I am training a classification model with 3 classes using a deep neural network. The classes have been resampled and balanced. I have around 600000 samples... equally distributed. The dataset is also ...
Fr_nkenstien's user avatar
0 votes
3 answers
73 views

Reduce false positives having imbalanced data

I'm using a DNN-48 having the following scenario: Features: 8 (48 at the end because I generate conditional sequences of 6 elements each) Classes: Y=0 (90%), Y=1 (10%) Precision and recall are good ...
Gabriel's user avatar
0 votes
1 answer
48 views

This simple python Feed forward Neural Network isn't learning. What am I doing wrong?

The backpropagation procedure is taken from the approach outlined in here. Here is the code, commented: ...
blundered_bishop's user avatar
0 votes
0 answers
31 views

Training loss is much higher than validation loss

I am trying to train a neural network with 2 hidden layers to perform a multi class classification of 3 different classes. There is a huge imbalance to the classes, with the distribution being around ...
joseph wong's user avatar
0 votes
0 answers
55 views

Why I am getting error in dataloader in defining a NN?

I am trying to write a NN. However I am getting error. Here is my Code: ...
Ali.A's user avatar
  • 57
0 votes
0 answers
7 views

Dynamic dosing recipe for accurate pH

I want to have a script for adjusting dosage of ingredients in each batch dynamically. Assuming that the requirement is to have a specific value of pH from each batch but with the variation of raw ...
Pittoon Hanvivatpong's user avatar
0 votes
0 answers
11 views

Questions regarding backward propagation

I am reading article regarding backward propagation https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ . Lets say if I follow the example in the article but using only 3 node, ...
CKT's user avatar
  • 1
0 votes
1 answer
29 views

Appropriate input size for nn.Embedding

I’m quite new to using Pytorch and deep learning. What size of unique categories of a categorical variable is appropriate for applying the nn.Embedding ideally (best practices)? for example, if a ...
Любовь Пономарева's user avatar
1 vote
0 answers
21 views

Why do I keep on getting ResourceExhaustedError while training on video data using CONV3D on tensorflow?

I'm encountering a memory allocation problem while training a deep learning model on my computer, which has a Core i9 10th Gen CPU, 64 GB of RAM, and an NVIDIA GTX 1660 Super with 6GB of VRAM. Despite ...
Ali Subhan's user avatar
1 vote
1 answer
51 views

Why my validation loss and accuracy decays over epochs?

Im trying to build 2 simple networks with cleaned dataset for tweets sentiment classification(0/1): one with all dense layers(binary bag of words) another with RNN layer(embedding layer). But it both ...
emily 's user avatar
  • 35
0 votes
1 answer
34 views

Tensorflow diagram for attention mechanism

I was reading the tutorial from tensorflow on the transformer model, however, when they explain the transformer model, they display such a picture : which I don't understand. What do the ingoing ...
edamondo's user avatar
  • 101
0 votes
0 answers
26 views

Multi input model tensorflow with fixed data as input

I am tring to implement the following architecture. alpha and beta are fixed matrices, they are matrices I want to input on every forward pass. Meaning for every batch they should be the same My ...
Ahmed Gado's user avatar
0 votes
0 answers
15 views

Making a NN closer to a linear regression?

It is possible to 'initialise' a gradient boosted model with a simpler model, such as linear regression, by manually setting the initial score. This seems to help reduce the discrepencies between the ...
Lucas Morin's user avatar
  • 2,567

1
2 3 4 5
88