Questions tagged [neural-network]
Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.
4,362
questions
2
votes
1
answer
124
views
How to analyze neural network quality in case of overfitting?
I have a Keras neural network that has images both as input and reference data.
My network demonstrates overfitting (for example, train accuracy is about 80% but test accuracy is only up to 70%) due ...
CommunityBot
- 1
0
votes
0
answers
15
views
Why isn't the validation data loss close to the test data loss?
First I set aside about 15% of my data as test data. Then, I used tensorflow.keras to create a relatively simple neural net model. Then I set the model.fit() parameter validation_split=0.2, so 20% of ...
- 1
1
vote
1
answer
72
views
How to find Neural Network ZOOs?
I have heard about the term Neural Network ZOOs, which are supposed to be repositories where there are a lot of pre-trained neural network models for many different applications, but I'm struggling to ...
CommunityBot
- 1
2
votes
1
answer
178
views
Doing a fine tuning after a transfer learning
I read about fine tuning and transfer learning for CNNs and was wondering if we can do fine tuning after using transfer learning on the same CNN? If so, will this increase the performance of the model ...
CommunityBot
- 1
0
votes
1
answer
235
views
Conv-2 CNN architecture - CIFAR-10
I have a CNN architecture for CIFAR-10 dataset which is as follows:
Convolutions: 64, 64, pool
Fully Connected Layers: 256, 256, 10
Batch size: 60
Optimizer: ...
CommunityBot
- 1
1
vote
2
answers
2k
views
What exactly is Gradient norm?
I found that there is no common resource and well defined definition for "Gradient norm", most search results are based on ML experts providing answers which involves gradient norm or papers ...
- 1
2
votes
1
answer
3k
views
Multilayer perceptron does not converge
I have been coding my own multi layer perceptron in MATLAB and it compiles without error. My training data features, x, has values from 1 to 360, and the training data output, y, has the value of $\...
CommunityBot
- 1
1
vote
1
answer
110
views
Does transfer learning make sense for small neural networks with only one or two hidden layers?
I am testing transfer learning on rather small neural networks with only two hidden layers of 20 neurons on tabular data.
None of my experiments yields any improvement over a basic neural network.
Is ...
CommunityBot
- 1
0
votes
1
answer
517
views
LSTM layer (keras) is causing all layers after it to constantly predict the same thing no matter the input
I have a model for OCR, which after 2-3 epochs gives the same output. When I predicted the values and looked at the output for each layer I realized that all layers after the 1st layer in the LSTM ...
CommunityBot
- 1
3
votes
1
answer
429
views
Neural Network regression negative performance
I have a problem with the performance of a multi layer perceptron regressor (neural network) and I cannot figure out why.
Task: I am trying to improve a time series prediction. I have predictions of a ...
CommunityBot
- 1
3
votes
1
answer
405
views
What is the meaning of an empty SHAP graph in Explainable AI?
Using Python, I created a neural network to perform predictions on a binary class dataset (e.g. will a passenger survive the Titanic?). I am using the SHAP package to explain individual predictions. ...
CommunityBot
- 1
2
votes
1
answer
928
views
Keras - Autoencoder different from Encoder + Decoder
I build a CNN 1d Autoencoder in Keras, following the advice in this SO question, where Encoder and Decoder are separated. My goal is to re-use the decoder, once the Autoencoder has been trained. The ...
CommunityBot
- 1
2
votes
1
answer
657
views
(Deep Learning) Backpropagation derivation from notes by Andrew NG
I am self-studying Andrew NG's deep learning course materials from the mcahine learning course (CS 229) of Stanford. The material is available here.
I have a question about the chain rule techniques ...
CommunityBot
- 1
0
votes
0
answers
19
views
Is there a Tensorflow built-in function to create a matrix from a single-layered feedforward neural network without activation functions?
In Tensorflow, I implemented a simple single-layer feedforward neural network with N inputs and N outputs without activation functions and biases. Simply, it is just a N-by-N matrix. Question: is ...
- 1
3
votes
1
answer
174
views
How is padding masking considered in the Attention Head of a Transformer?
For purely educational purposes, my goal is to implement basic Transformer architecture from scratch. So far I focused on the encoder for classification tasks and assumed that all samples in a batch ...
CommunityBot
- 1
0
votes
1
answer
194
views
Build autoencoder for single matrix with integer numbers
Can you please tell me how to build an autoencoder with a single matrix(4,4) with integer numbers?
I want to build an autoencoder for the below-mentioned data. I don't know whether I should convert ...
CommunityBot
- 1
0
votes
1
answer
1k
views
How is calculated the error with multiple output neurons in neural network?
Machine Learning books generally explains that the error calculated for a given sample $i$ is:
$e_i = y_i - \hat{y_i}$
Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
CommunityBot
- 1
2
votes
1
answer
106
views
Why the test accuracy showing some odd behaviour in comparison to train accuracy?
I am currently training an ANN using Sequential(a class from Keras API within tensorflow), and I am optimizing the model's architecture and came across something I have not seen before.
The graph of ...
- 161
0
votes
0
answers
4
views
Accuracy Drop in ViT with Patch Embedding: Investigating the Impact of Added Convolutional Layers
I'm currently working on incorporating a patch embedding layer into my Vision Transformer (ViT). I've defined this layer using four 2D convolutional and initialized it with a normal distribution. The ...
3
votes
3
answers
220
views
Is there a rule of thumb when designing neural network in deep reinforcement learning?
In deep learning, we can assess model's performance with loss function value and improve model's performance with K-fold cross-validation and so on. But how can we design and tune neural network used ...
CommunityBot
- 1
1
vote
1
answer
177
views
Building a text classification model from scratch
I am a beginner in data science and machine learning techniques. I would need to build a model that allows me to classify texts based on sentiment analysis.
Right now I only have the text and they ...
CommunityBot
- 1
0
votes
2
answers
7k
views
Validation Accuracy Not Changing
As the title states, my validation accuracy isn't changing when I try to train my model. I've built an NVIDIA model using tensorflow.keras in python. I have absolutely no idea what's causing the issue....
CommunityBot
- 1
0
votes
0
answers
11
views
Packet data needed fot the model
I want to use this data set https://huggingface.co/datasets/rdpahalavan/network-packet-flow-header-payload for network attack classification
Here is packet structure from the README file.
Assuming I ...
- 101
1
vote
2
answers
84
views
Will my validation loss eventually go down?
I'm currently training a binary classifier that takes in 2 inputs, and outputs which object it thinks is "better."
I have an absolutely massive dataset, about 2 trillion records, and I'm ...
CommunityBot
- 1
0
votes
1
answer
70
views
Why am a getting wrong prediction when combining two list of samples, which individually gives correct prediction?
So I am coding in Python. I have to set of samples. Set1 contains samples of class A and the other set, Set2 contains samples of class B. These samples taken are a part of the training dataset. When I ...
CommunityBot
- 1
2
votes
3
answers
88
views
Riskscore creation on Numerical Data
I am working to create a Risk score on data where I have variables -
Invested_amount, Profit Amount, Age of Account in days, Total Trading Transactions, Profit per Transaction & Investment per ...
CommunityBot
- 1
0
votes
1
answer
254
views
Segmentation 3D Unet checkerboard artifacts in slices above and below segmentation voxels
I suppose an image is worth too many words, so here is the image:
As you can see, in the middle where there are voxels to be segmented, no artifacts are present. Whereas on the top and bottom I get a ...
CommunityBot
- 1
2
votes
1
answer
67
views
How to weigh feature array
I have a feature array of around 4000 elements, extracted from one source. On this array I've extracted 7 more feature from other source and now I basically have a 4007 feature array from each data ...
CommunityBot
- 1
2
votes
1
answer
485
views
Custom connections between layers Keras
I would like to manually define connections in neural network between layers using TensorFlow and keras with Python. By default connections are beween all pairs of neurons. I need to make connections ...
CommunityBot
- 1
5
votes
2
answers
4k
views
Batching in Recurrent Neural Networks (RNNs) when there is only a single instance per time step?
I have scoured the internet and books, but everything seems to use num_steps and batch_size or similar terms interchangeably and ...
- 1
0
votes
1
answer
16
views
Holding batch size constant, will a bigger dataset consume more GPU memory?
If you hold (mini) batch size constant (as well as everything else) but increase the number of examples (and therefore the number of training iterations), should you expect a (significant) increase in ...
- 101
1
vote
1
answer
236
views
Building a prediction model for dynamic coordinates and later categorize as binary classification
Project Summary
I have a website (academic project) that records mouse movements such as click, mouse up, mouse down, etc. It records the coordinates for each event on a given web page from a visitor....
CommunityBot
- 1
0
votes
2
answers
91
views
Predicting a relative distribution between the available instances
I am working on a model that is supposed to predict how a given order volume distributes over the available articles in a retail scenario. For simplicity's sake, let's say I'm a retailer that buys ...
CommunityBot
- 1
2
votes
1
answer
143
views
How Does Cross-Entropy Work With Softmax Activation Function?
I found online that the derivative of a cross-entropy activation function with a softmax activation is (output - expected), which had me very confused. If for example, the expected value is 1, and ...
CommunityBot
- 1
3
votes
1
answer
92
views
Train a classifier for a game with feedback on chosen move instead of true labels
I'm having some trouble describing in one line what I want, which is probably why I haven't had much luck with Google.
Say I have a game like 2048 where the possible actions each step are fixed (and ...
CommunityBot
- 1
39
votes
3
answers
35k
views
Why use both validation set and test set?
Consider a neural network:
For a given set of data, we divide it into training, validation and test set. Suppose we do it in the classic 60:20:20 ratio, then we prevent overfitting by validating the ...
- 258
2
votes
2
answers
179
views
Neural Network stacking layers
I was watching Andrew Ng's video on ResNets, and he mentioned that "And in theory, as you make a neural network deeper, it should only do better and better on the training set."
Here is my ...
CommunityBot
- 1
1
vote
2
answers
1k
views
Understanding Embeddings input and output sizes
I have been trying for a while to understand the dimensionality of embeddings in neural networks and I think that finally things have clicked in my brain. However, I would love to check whether or not ...
CommunityBot
- 1
2
votes
1
answer
353
views
One-hot & interaction one-hot on multiple categorical
I was wondering if there is any value to creating combined features out of multiple categorical variables when the individual categorical variables are already one-hot encoded?
Simple example: there ...
CommunityBot
- 1
1
vote
1
answer
590
views
"RuntimeWarning: overflow[...]" in TensorFlow in juypter notebook
I'm implementing a captcha solver in tensorflow (also using ipython/jupyter notebook). After adding more layers to my model it now prints this message in a red box which I don't understand. What does ...
CommunityBot
- 1
1
vote
2
answers
161
views
Neural network reaching local optima
I was recently trying to train a convolutional neural network to classify people as Hispanic or white (for learning purposes). I couldn't find a good dataset of just those two races, so I had to ...
CommunityBot
- 1
2
votes
1
answer
186
views
Showing standard deviation for training curve
I am training a neural network and I wanted to plot the evolution of different metrics (MSE…) during training. To get an idea of the variations between between different trainings, I am using several ...
CommunityBot
- 1
1
vote
2
answers
150
views
Anomaly detection for time series data with only positive samples?
I'm having a time series ECG dataset. I want to do anomaly detection (anything different from normal ECG should be abnormal).
The point is I'm having only positive samples with very few negative ...
CommunityBot
- 1
2
votes
1
answer
76
views
2
votes
2
answers
190
views
How to interpret sudden jumps of improvement in training error vs. iteration?
In the Residual learning paper by He et al., there are a number of plots of training/test error vs. backprop iteration. I've only ever seen "smooth" curves on these plots, while in this paper's ...
CommunityBot
- 1
2
votes
3
answers
409
views
Training neural network to emulate a hash function
A hash function takes an input, performs a set of complex operations and then produces an output. For my purposes the output from the function will always be the same for any given input.
I remember ...
- 101
2
votes
1
answer
615
views
What are the equations involved in calculation of the parameters of embedding layer?
I'm trying to perform sentiment analysis on some data using keras.I'm using embedding layer and then LSTM. I know that embedding layer decreases the sparsity of the one hot encodings of the words and ...
CommunityBot
- 1
0
votes
1
answer
154
views
What do graphs of signal vs background neural network outputs represent?
For example, this image shows the output of a neural network I assume but I am not sure how the output is not symmetric. So if the neural network gives a .6 for signal it should give a .4 for ...
CommunityBot
- 1
0
votes
0
answers
76
views
Neural Net not able to learn simple analytical equation
I am currently making my first attempts with Pytorch. I am trying to solve a simple equation with a neural net. Analytically solved, the result of my neural net shall look like this:
$$
y = \frac{x_5}{...
- 1
1
vote
2
answers
449
views
Metrics values are equal while training and testing a model
I'm working on a neural network model with python using Keras with TensorFlow backend. Dataset contains two sequences with a result which can be 1 or 0 and positives to negatives ratio in dataset is 1 ...
CommunityBot
- 1