Questions tagged [neural-network]
Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.
4,300
questions
0
votes
0
answers
11
views
Accelerated learning when wrapping layers in a class
I am implementing a VGG-like network using Pytorch 1.13.1 (python=3.7.12) for image classification on the CINIC-10 dataset. The following two implementations turn out to have very different training ...
- 1
0
votes
1
answer
18
views
GPT-2 architecture question
I am currently working on a NLP model that compares two comments and determines which one would be more popular. I have already came up with an architecture - it will be based on GPT-2. But now I am ...
0
votes
1
answer
26
views
Understanding correlation - Machine Learning
I am experimenting a project on identifying cancer or not - Binary classification
The dataset has many columns. Here, I added correlation values between few input columns and the target column[cancer/...
- 21
0
votes
0
answers
8
views
Problem of constant shift in prediction for neural network regression model with gradient-domain loss function
I'm training a regression model using neural network which is trained on MSE of both output and spatial gradient of output. With some simplification, the model is:
$$
y = f(\mathbf{x};\theta)
$$
where ...
- 1
1
vote
1
answer
19
views
How to bias a neural network towards one category in binary classification?
I have a basic sequential neural network built with TensorFlow.
...
0
votes
0
answers
11
views
Siamese Neural network inputs
A currently task involves the classification of bacteria as antibiotic susceptible and antibiotic resistant.
I have 4 data sets: treated resistant, treated susceptible, untreated resistant and ...
0
votes
0
answers
17
views
How to properly perform K-fold cross validation with train, eval and test sets while building NN model?
Intro
I am training simple neural network and want to properly evaluate my model. It is not entirely clear to me, which dataset should I divide into folds in K-fold CV working with train, eval and ...
- 103
0
votes
1
answer
16
views
How to split a single feature vector into a layer of 2 neurons
Given an array x = [1, 2, 3, ...] , I want to split each sample x[i] into 2 neurons.
My idea was to initialize a variable ...
- 122
0
votes
0
answers
8
views
In "Show, attend and tell", why do the attention weights get multiplied with the features to form the context vector?
The attention weights are formed through the last hidden state of the LSTM and the feature map from some kind of image encoder (in my case resnet so the features are in the form of 14x14x2048). They ...
- 101
0
votes
0
answers
20
views
Workflow for improving and comparing deep learning models
Say I have the most basic neural network that performs some task (eg Keras sequential model with one hidden layer, used to binary classification) and a list of ideas how one could improve it (like: ...
- 103
0
votes
0
answers
8
views
Transfer learning applicability for Psychology experimental research
This is my first question so please be gentle!
I am a Psychologist building a predictive model using experimental data and I want to know how I can do this using limited training data from part of my ...
1
vote
1
answer
20
views
Example of a 2D dataset and a classifier stuck at local minimum
We always hear about neural networks getting stuck at local minima, but I cannot visualize one. Can you please give me some examples?
I am not looking for something like below picture and a neural ...
- 111
0
votes
2
answers
20
views
Using time serie to predict another variable
I would like to analyse head rotation data in space. For this I measured at 15HZ the rotation around the X, Y and Z angles for a little more than ten minutes. I would like to use these movements to ...
0
votes
0
answers
36
views
Bugs in the backpropagation algorithm in Python
I've been trying to create a simple Neural Network from scratch with a backpropagation algorithm to predict the next number based on 3 previous numbers. But for some reasons, MSE(Mean Squared Error) ...
2
votes
1
answer
45
views
Multiple classes present in one-hot encoding
When dealing with classification for multiple classes present in the same sample, can the output layer have the form of one-hot encoding, but instead of only one hot, have multiple?
That is, in case ...
- 23
0
votes
1
answer
33
views
Which chess notation to feed neural network: FEN or PGN?
I am trying to build a chess AI with a neural network. To learn about how neural networks work and refresh my programming experience. I have some experience with classifiers but not yet with neural ...
- 123
0
votes
1
answer
107
views
Pytorch mat1 and mat2 shapes cannot be multiplied
The error message shows
RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x32768 and 512x256)
I have built the following model:
...
- 27
0
votes
1
answer
42
views
Decision boundary of an neural network
Starting with a).
For the first unit: 0 * x1 + 1 * x2 + 1 > 0 (0, because the threshold is 0) which is the same as x2+1 > 0.
For the second unit: x1 * 1 + x2 * 0 + 1 > 0 (0, because the ...
- 11
0
votes
1
answer
15
views
Force network to weigh specific variables during learning
I have a pandas data frame containing around 100000 observations of plant species and their age with additional numerical predictors (climate). I used tensorflow ...
- 53
0
votes
0
answers
4
views
Is a conv transpose layer equivalent to a padding layer and regular conv layer
Is a 2d convolution transpose layer equivalent to a upsampling layer that inserts 0s between rows and columns, then a regular 2d convolution layer? If so, why is it usually not implemented as such (i....
- 101
0
votes
0
answers
12
views
What models are able to handle variable input lengths?
I am tasked with creating a classifier that is able to predict whether an item will be returned.
This is supposed to not only happen on the basis of an individual item, but on all other items within ...
0
votes
0
answers
18
views
Need help with improving validation loss and model overfitting/underfitting
I am using Ensemble PyTorch to train a voting classifier. My dataset includes around 60k records. I trained a Neural Network with Cross-entropy loss. Below is my model architecture
...
- 1
0
votes
0
answers
10
views
Back Propagation on arbitrary depth network with ReLu
I am implementing a neural network of arbitrary depth with an arbitrary number of nodes on each depth. My forwards propagation thus looks like this (For 2 hidden layers)
...
0
votes
0
answers
16
views
Teaching the model on more than one dataframe/dataset
I am trying to write a thesis on oil pipe leakage detection. The aim is to predict the size and location of the leak. My problem is that I can only run single simulations using a software called OLGA ...
0
votes
1
answer
19
views
What does Codex take as tokens?
The typical default for neural networks in natural language processing has been to take words as tokens.
OpenAI Codex is based on GPT-3, but also deals with source code. For source code in general, ...
- 115
1
vote
0
answers
12
views
In WGAN paper, why does clipping weights approximate Lipschitz function?
In Wasserstein GAN, it's explained that maximizing a certain formula over a set of K-Lipschitz functions approximates the 1-Wasserstein distance and they model the functions as NNs. That much I ...
- 11
2
votes
1
answer
15
views
Im looking for good neurons silmilarity metric
Recently I managed to create simple neural network visualization, to help to understand how neural network works on the signal level. I also wanted to arrange neurons by similarity cause I was ...
- 21
0
votes
0
answers
6
views
Optimize wordembedding and neural network at the same time
I have a lot of (domain)-specific text that I want to classify into 100+ categories.
I want to train a wordembedding (FastText) and use that in conjuction with a CNN, thus I'm running into the problem ...
- 440
0
votes
0
answers
10
views
How do i calculate 1 iteration of the backpropagation algorithm on this exercise
Hello, i'm currently learning Neural Networks, so have a lot of doubts on how do i calculate the weights after one iteration.
The step function for neuron 1 is:y1 = step(x1 + x2 -1.5)
The step ...
0
votes
0
answers
10
views
Is there a way to use CNN to separate/cluster the images into N clusters without online learning or only very mild online learning?
There are lots of examples to use CNN as classifier to separate images into known classes. There also lots of examples to use CNN as encoder and generate embedding to check the similarity of objects.
...
- 101
-1
votes
1
answer
29
views
Classifying the sum of two inputs as even or odd - Model always stuck at 50% accuracy for training and test data
I am creating a simple feed forward to classify if the sum of two inputs as even or odd.
I cannot change the input structure (has to be two nodes), and output structure (two nodes as well, one for ...
0
votes
0
answers
9
views
I am creating an multilayer LSTM model from scratch and everything seems to be mathematically correct however the model refuses to learn
I am creating the LSTM with just numpy and plotting the loss with pyplot. I have checked the derivatives again and again however have not found a mistake. The entire code with the main function can be ...
0
votes
0
answers
14
views
Combining 2 losses for 2 different tasks and training the networks in Keras
I have to implement a communication model consisting of a mapper , channel , detector and demapper blocks. The mapper , detector and demapper blocks are neural networks and channel is a AWGN channel.
<...
- 1
0
votes
1
answer
21
views
Machine learning / statistical model of a deterministic process: how large must my training set be to ensure almost perfect accuracy?
This may be a silly question, but if I got a deterministic process, for instance, a function (in the mathematical sense) that happens to be computationally expensive to evaluate, and I decided to ...
- 103
0
votes
1
answer
30
views
Classification of a noisy data
What method can be used to classify data in the following example?
There is a table (hundreds of strings and hundreds of columns). Several columns in this table uniquely allow you to classify each row:...
- 1
0
votes
0
answers
11
views
How to shape the input for Temporal Convolutional Networks
Consider a normal time series coming from stock prices, assume for simplicity it's several thousand data points. So basically I have a time series of prices $\{x_i\}_{i=0}^n$ of $n$ data points.
I ...
- 101
0
votes
0
answers
11
views
Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?
I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it ...
- 101
0
votes
0
answers
16
views
Understanding perceptron learning algorithm
I was revisiting perceptron learning algorithm. The wikipedia page gives the algorithm as follows:
Initialize the weights to 0 or a small random value.
For each example $j$ in our training set $D$, ...
- 99
0
votes
0
answers
12
views
CNN sharing weights in feature map
what do they mean when they say all
neurons in a channel share weights with one another? Do they mean that in a chanel or a featue map the weights are the same ?
0
votes
0
answers
15
views
Any guidance on designing neural network to fit multiplication?
I want to explore the ability of NN to fit multiplication, so I design a dataset and fit it using LSTM.
The dataset:
X: samples x 60 x 1; the timestep is 60, the feature dimension is 1
Y: y[i] = corr(...
0
votes
1
answer
21
views
Can I change the number of inputs to a keras model while preserving the trained existing weights
I have a simple Sequential keras model with 150 Inputs.
Some of these are simply OneHotEncoded values. Now I would like to add more options to the OneHotEncoder.
As an example: I previously had Blue, ...
- 3
0
votes
0
answers
17
views
How to visualise a feature map or filter in a group equivariant convolutional neural network
I'm reading an article called "Group Equivariant Convolutional Networks" by T. Cohen and M. Welling (https://arxiv.org/abs/1602.07576) and I'm having some problems understanding one of their ...
- 1
0
votes
1
answer
15
views
Train/val/test approach for hyperparameter tuning
When looking to train a model, does it make sense to have a 60-20-20 train val test split, first hyper parameter tuning over the training dataset, using the validation set, picking the best model. ...
- 91
0
votes
0
answers
16
views
What is semi-ground truth?
What is semi-ground truth and how it is different from ground truth? Can you explain with an example?
- 1
1
vote
1
answer
20
views
How to determine which combinations of parameters to include in GridSearchCV
I am using MLPClassifier from sklearn and I would like to tune it with GridSearchCV. But I don't know which set of values to include for hidden_layer_sizes, max_iter, activation, solver, etc. How can ...
- 11
0
votes
0
answers
25
views
Is this overfitting? (generative model)
I am working with a generative method, and the network seems to perform well on training data and slightly less well on test data, but the generated data is somehow significantly worse than either of ...
- 31
0
votes
0
answers
15
views
Neural networks and input filters
In my use case scenario, I have a neural network that should filter the input and pass a specific value of the input array to the output.
In particular, let's define the input as:
...
- 1
0
votes
0
answers
9
views
OR gate perceptron in plain python - Loss won't converge
I am coding a perceptron from scratch just out of curiosity in plain python for OR gate, but a loss won't converge.
...
- 101
0
votes
0
answers
10
views
Weighting loss functions for multi task learning
I am training a multi-task neural network which is predicting a binary target variable, an 18-class target, and a 17-class target. I am calculating the cross-entropy loss for each task, then summing ...
- 1
0
votes
0
answers
12
views
Relative changes instead absolute values in LSTM training
I am reading some papers about glucose time series prediction and I have noticed that some of them propose LSTM models that use relative changes between two measures.
For example, if $$ glucose(t)=60, ...
- 1