Questions tagged [neural-network]
Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.
4,333 questions
0
votes
1
answer
315
views
while re-training a pre-trained model, I'm facing this issue RuntimeError: You must compile your model before using it
model summary:
RuntimeError: You must compile your model before using it.
It says that the model needs to be compiled. But as far i know, if i compile a model, all the previous trained data will be ...
CommunityBot
- 1
2
votes
1
answer
2k
views
2
votes
1
answer
3k
views
Multilayer perceptron does not converge
I have been coding my own multi layer perceptron in MATLAB and it compiles without error. My training data features, x, has values from 1 to 360, and the training data output, y, has the value of $\...
CommunityBot
- 1
0
votes
1
answer
1k
views
Embedding layer before LSTM layer
I am toying around with a clustering and churn prediction framework, cluschurn which they deployed in production at Snap, Inc. In their research paper, paper_link, they use 14 days of user data and ...
CommunityBot
- 1
1
vote
1
answer
76
views
How does time needed for training differ between different batch sizes?
I've constructed a CNN in Python using Numpy, which is trained with mini batch gradient descent for MNIST digit classification. When training with a batch size of 1, the time needed for 5 epochs is ...
CommunityBot
- 1
1
vote
1
answer
341
views
More weightage to a categorical feature for an Autoencoder model
I am using autoencoder for anomaly detection. I don't have any labels already and so its unsupervised. If I have categorical variables, I usually one hot encode before giving it to the model. I would ...
CommunityBot
- 1
2
votes
1
answer
188
views
Doing a fine tuning after a transfer learning
I read about fine tuning and transfer learning for CNNs and was wondering if we can do fine tuning after using transfer learning on the same CNN? If so, will this increase the performance of the model ...
CommunityBot
- 1
2
votes
2
answers
205
views
Neural Network stacking layers
I was watching Andrew Ng's video on ResNets, and he mentioned that "And in theory, as you make a neural network deeper, it should only do better and better on the training set."
Here is my ...
CommunityBot
- 1
0
votes
0
answers
7
views
How does innovation deal with the problem of different connections in NEAT?
How does innovation deal with the problem of different connections in NEAT?
An example makes it clearer: the neural network has two outputs 3 and 4, they are connected according to the scheme 10 -> ...
0
votes
1
answer
260
views
NAN in keras neural network results
I am creating a neural network simple architecture. But I keep getting NAN in result, cant figure out why, below is my code.
...
CommunityBot
- 1
2
votes
2
answers
4k
views
How to properly save and load an intermediate model in Keras?
I'm working with a model that involves 3 stages of 'nesting' of models in Keras.
Conceptually the first is a transfer learning CNN model, for example MobileNetV2. (Model 1) This is then wrapped by a ...
CommunityBot
- 1
8
votes
2
answers
612
views
How does one derive the modified tanh activation proposed by LeCun?
In "Efficient Backprop" (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf), LeCun and others propose a modified tanh activation function of the form:
$$ f(x) = 1.7159 * tanh(\frac{2}{3}*x) $$
...
- 141
1
vote
0
answers
15
views
Neural Network Overfitting on Linearly Separable Dataset
Please let me know if this question is not proper to ask here
For context, I have a dataset regarding to tiktok user engagement. The predicted variable is binary, either 'claim' or 'opinion'. From ...
- 11
2
votes
1
answer
551
views
Custom connections between layers Keras
I would like to manually define connections in neural network between layers using TensorFlow and keras with Python. By default connections are beween all pairs of neurons. I need to make connections ...
CommunityBot
- 1
77
votes
6
answers
155k
views
Cross-entropy loss explanation
Suppose I build a neural network for classification. The last layer is a dense layer with Softmax activation. I have five different classes to classify. Suppose for a single training example, the <...
- 116
1
vote
1
answer
386
views
Derivation of dz[1] for backpropagation
Can anyone mathematically prove this equation given the values of $dz^{[2]}$, $W^{[2]}$, $z^{[1]}$ and the activation function $g^{[1]}$
$dz^{[1]} = w^{[2]T}dz^{[2]} * g^{[1]'}(z^{[1]})$
CommunityBot
- 1
0
votes
1
answer
403
views
Segmentation 3D Unet checkerboard artifacts in slices above and below segmentation voxels
I suppose an image is worth too many words, so here is the image:
As you can see, in the middle where there are voxels to be segmented, no artifacts are present. Whereas on the top and bottom I get a ...
CommunityBot
- 1
1
vote
1
answer
142
views
Does transfer learning make sense for small neural networks with only one or two hidden layers?
I am testing transfer learning on rather small neural networks with only two hidden layers of 20 neurons on tabular data.
None of my experiments yields any improvement over a basic neural network.
Is ...
CommunityBot
- 1
3
votes
3
answers
246
views
Is there a rule of thumb when designing neural network in deep reinforcement learning?
In deep learning, we can assess model's performance with loss function value and improve model's performance with K-fold cross-validation and so on. But how can we design and tune neural network used ...
CommunityBot
- 1
2
votes
3
answers
124
views
Riskscore creation on Numerical Data
I am working to create a Risk score on data where I have variables -
Invested_amount, Profit Amount, Age of Account in days, Total Trading Transactions, Profit per Transaction & Investment per ...
CommunityBot
- 1
1
vote
1
answer
82
views
How to find Neural Network ZOOs?
I have heard about the term Neural Network ZOOs, which are supposed to be repositories where there are a lot of pre-trained neural network models for many different applications, but I'm struggling to ...
CommunityBot
- 1
2
votes
1
answer
90
views
Weird distribution of neural network outputs
I've faced an unusual behavior during training a neural network.
The problem is to predict if a sample of 1st class or 2nd class. (2-class classification). Classes are imbalanced (~ 5 / 95). I use ...
CommunityBot
- 1
0
votes
3
answers
4k
views
Why does hyperparameter tuning occur on validation dataset and not at the very beginning?
Despite doing/using it a few times, I'm still slightly confused by the use of a validation set for hyper parameter tuning.
As far as I can tell, I choose a model, train it on training data, assess ...
CommunityBot
- 1
1
vote
2
answers
891
views
What is preferred upsampling or zero padding?
When training a CNN one option is either to zero pad an image to make it bigger or upsample it. When should I choose each one?
What criteria is leveraged for choosing a method?
CommunityBot
- 1
3
votes
1
answer
596
views
Neural Network regression negative performance
I have a problem with the performance of a multi layer perceptron regressor (neural network) and I cannot figure out why.
Task: I am trying to improve a time series prediction. I have predictions of a ...
CommunityBot
- 1
6
votes
1
answer
7k
views
Why maximize ELBO in the variational autoencoder?
For a variational autoencoder, we have that:
$$\mathscr{L}(x,\theta,\phi) := \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_{\theta}(x|z)] -KL[q_{\phi}(z|x) ||p(z)] $$
This is called the variational lower ...
- 163
1
vote
1
answer
37
views
Examples of uses of neural networks where you can rigorously define desired properties of the solution?
Neural networks are often used to solve problems where we can't rigorously define what properties the desired solution should have, e.g. you can't define what a "picture of a cat" is and so ...
CommunityBot
- 1
2
votes
1
answer
981
views
Keras - Autoencoder different from Encoder + Decoder
I build a CNN 1d Autoencoder in Keras, following the advice in this SO question, where Encoder and Decoder are separated. My goal is to re-use the decoder, once the Autoencoder has been trained. The ...
CommunityBot
- 1
0
votes
1
answer
76
views
Why am a getting wrong prediction when combining two list of samples, which individually gives correct prediction?
So I am coding in Python. I have to set of samples. Set1 contains samples of class A and the other set, Set2 contains samples of class B. These samples taken are a part of the training dataset. When I ...
CommunityBot
- 1
2
votes
1
answer
729
views
(Deep Learning) Backpropagation derivation from notes by Andrew NG
I am self-studying Andrew NG's deep learning course materials from the mcahine learning course (CS 229) of Stanford. The material is available here.
I have a question about the chain rule techniques ...
CommunityBot
- 1
1
vote
0
answers
22
views
How to choose a neural network architecture?
How to choose a neural network architecture?
Examples:
«What if I need to translate words?»
«Generate text, images?»
«Play a regular game?»
«Play a game that changes depending on the player's actions, ...
2
votes
1
answer
3k
views
Why does Faster R-CNN use SGD optimizer instead of Adam?
I just start learning Faster R-CNN and I have some doubts about the optimizer of this network. In my understanding, Adam optimizer performs much better than SGD in a lot of networks. However, the ...
CommunityBot
- 1
0
votes
1
answer
2k
views
How is calculated the error with multiple output neurons in neural network?
Machine Learning books generally explains that the error calculated for a given sample $i$ is:
$e_i = y_i - \hat{y_i}$
Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
CommunityBot
- 1
1
vote
3
answers
489
views
How to use text as an input for a neural network - regression problem? How many likes/claps an article will get
I am trying to predict the number of likes an article or a post will get using a NN.
I have a dataframe with ~70,000 rows and 2 columns: "text" (predictor - strings of text) and "likes&...
CommunityBot
- 1
2
votes
0
answers
20
views
LSTM produces a straight line for predictionsout of range data
I have this problem:
I am trying to predict daily temperatures. I have data of 30 years, and I am using this neural network:
...
- 21
1
vote
2
answers
112
views
Will my validation loss eventually go down?
I'm currently training a binary classifier that takes in 2 inputs, and outputs which object it thinks is "better."
I have an absolutely massive dataset, about 2 trillion records, and I'm ...
CommunityBot
- 1
1
vote
0
answers
29
views
ML. How to make a neural network remember the context and data?
I want the neural network to be able to remember, but a perceptron can only remember something during training, but I want the neural network to adapt to new conditions without retraining, for example,...
0
votes
1
answer
362
views
Sliding window approach using SVR & LightGBM
I'm working on a multivariate time series forecast using a couple of ML algorithms (Neural Networks, Support Vector Machines & Gradient boosting algorithms). I need to measure the performance of ...
CommunityBot
- 1
3
votes
1
answer
331
views
How is padding masking considered in the Attention Head of a Transformer?
For purely educational purposes, my goal is to implement basic Transformer architecture from scratch. So far I focused on the encoder for classification tasks and assumed that all samples in a batch ...
CommunityBot
- 1
0
votes
3
answers
871
views
What are some general tips to improve my MNIST classifier?
I have built a CNN from scratch in python using Numpy, to tackle the MNIST hand-written digit recognition problem. It's composed out of a convolutional layer (3 3x3 filters), a maxpooling layer (2x2 ...
CommunityBot
- 1
1
vote
1
answer
59
views
wierd neural network approache
I'm working on a problem where I need to create a neural network to optimize the seating arrangement for 24 unique individuals in a 6x4 grid, minimizing conflicts between adjacent (up,down,left,right) ...
CommunityBot
- 1
0
votes
1
answer
286
views
Build autoencoder for single matrix with integer numbers
Can you please tell me how to build an autoencoder with a single matrix(4,4) with integer numbers?
I want to build an autoencoder for the below-mentioned data. I don't know whether I should convert ...
CommunityBot
- 1
1
vote
1
answer
608
views
"RuntimeWarning: overflow[...]" in TensorFlow in juypter notebook
I'm implementing a captcha solver in tensorflow (also using ipython/jupyter notebook). After adding more layers to my model it now prints this message in a red box which I don't understand. What does ...
CommunityBot
- 1
2
votes
1
answer
372
views
MLP batch iteration in python
I'm using the MLPRegressor in sklearn to train a network with approx 1000 inputs and a continuous output variable. Essentially, the issue is one of image classification (1000 pixels) with the ...
CommunityBot
- 1
1
vote
1
answer
184
views
What do graphs of signal vs background neural network outputs represent?
For example, this image shows the output of a neural network I assume but I am not sure how the output is not symmetric. So if the neural network gives a .6 for signal it should give a .4 for ...
CommunityBot
- 1
1
vote
1
answer
123
views
How to handle overfitting in the following classification case
The confusion matrix is as below :-
[[ 0 0 5 1 0 0]
[ 0 0 19 14 0 0]
[ 0 0 217 151 0 0]
[ 0 0 84 282 0 0]
[ 0 0 6 111 0 0]
[ 0 0 0 10 0 0]]
...
CommunityBot
- 1
3
votes
1
answer
96
views
Train a classifier for a game with feedback on chosen move instead of true labels
I'm having some trouble describing in one line what I want, which is probably why I haven't had much luck with Google.
Say I have a game like 2048 where the possible actions each step are fixed (and ...
CommunityBot
- 1
1
vote
2
answers
173
views
Neural network reaching local optima
I was recently trying to train a convolutional neural network to classify people as Hispanic or white (for learning purposes). I couldn't find a good dataset of just those two races, so I had to ...
CommunityBot
- 1
2
votes
1
answer
697
views
What are the equations involved in calculation of the parameters of embedding layer?
I'm trying to perform sentiment analysis on some data using keras.I'm using embedding layer and then LSTM. I know that embedding layer decreases the sparsity of the one hot encodings of the words and ...
CommunityBot
- 1
1
vote
1
answer
215
views
Building a text classification model from scratch
I am a beginner in data science and machine learning techniques. I would need to build a model that allows me to classify texts based on sentiment analysis.
Right now I only have the text and they ...
CommunityBot
- 1