Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

Accelerated learning when wrapping layers in a class

I am implementing a VGG-like network using Pytorch 1.13.1 (python=3.7.12) for image classification on the CINIC-10 dataset. The following two implementations turn out to have very different training ...
  • 1
0 votes
1 answer
18 views

GPT-2 architecture question

I am currently working on a NLP model that compares two comments and determines which one would be more popular. I have already came up with an architecture - it will be based on GPT-2. But now I am ...
0 votes
1 answer
26 views

Understanding correlation - Machine Learning

I am experimenting a project on identifying cancer or not - Binary classification The dataset has many columns. Here, I added correlation values between few input columns and the target column[cancer/...
0 votes
0 answers
8 views

Problem of constant shift in prediction for neural network regression model with gradient-domain loss function

I'm training a regression model using neural network which is trained on MSE of both output and spatial gradient of output. With some simplification, the model is: $$ y = f(\mathbf{x};\theta) $$ where ...
  • 1
1 vote
1 answer
19 views

How to bias a neural network towards one category in binary classification?

I have a basic sequential neural network built with TensorFlow. ...
0 votes
0 answers
11 views

Siamese Neural network inputs

A currently task involves the classification of bacteria as antibiotic susceptible and antibiotic resistant. I have 4 data sets: treated resistant, treated susceptible, untreated resistant and ...
0 votes
0 answers
17 views

How to properly perform K-fold cross validation with train, eval and test sets while building NN model?

Intro I am training simple neural network and want to properly evaluate my model. It is not entirely clear to me, which dataset should I divide into folds in K-fold CV working with train, eval and ...
0 votes
1 answer
16 views

How to split a single feature vector into a layer of 2 neurons

Given an array x = [1, 2, 3, ...] , I want to split each sample x[i] into 2 neurons. My idea was to initialize a variable ...
  • 122
0 votes
0 answers
8 views

In "Show, attend and tell", why do the attention weights get multiplied with the features to form the context vector?

The attention weights are formed through the last hidden state of the LSTM and the feature map from some kind of image encoder (in my case resnet so the features are in the form of 14x14x2048). They ...
  • 101
0 votes
0 answers
20 views

Workflow for improving and comparing deep learning models

Say I have the most basic neural network that performs some task (eg Keras sequential model with one hidden layer, used to binary classification) and a list of ideas how one could improve it (like: ...
0 votes
0 answers
8 views

Transfer learning applicability for Psychology experimental research

This is my first question so please be gentle! I am a Psychologist building a predictive model using experimental data and I want to know how I can do this using limited training data from part of my ...
1 vote
1 answer
20 views

Example of a 2D dataset and a classifier stuck at local minimum

We always hear about neural networks getting stuck at local minima, but I cannot visualize one. Can you please give me some examples? I am not looking for something like below picture and a neural ...
0 votes
2 answers
20 views

Using time serie to predict another variable

I would like to analyse head rotation data in space. For this I measured at 15HZ the rotation around the X, Y and Z angles for a little more than ten minutes. I would like to use these movements to ...
0 votes
0 answers
36 views

Bugs in the backpropagation algorithm in Python

I've been trying to create a simple Neural Network from scratch with a backpropagation algorithm to predict the next number based on 3 previous numbers. But for some reasons, MSE(Mean Squared Error) ...
2 votes
1 answer
45 views

Multiple classes present in one-hot encoding

When dealing with classification for multiple classes present in the same sample, can the output layer have the form of one-hot encoding, but instead of only one hot, have multiple? That is, in case ...
  • 23
0 votes
1 answer
33 views

Which chess notation to feed neural network: FEN or PGN?

I am trying to build a chess AI with a neural network. To learn about how neural networks work and refresh my programming experience. I have some experience with classifiers but not yet with neural ...
  • 123
0 votes
1 answer
107 views

Pytorch mat1 and mat2 shapes cannot be multiplied

The error message shows RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x32768 and 512x256) I have built the following model: ...
  • 27
0 votes
1 answer
42 views

Decision boundary of an neural network

Starting with a). For the first unit: 0 * x1 + 1 * x2 + 1 > 0 (0, because the threshold is 0) which is the same as x2+1 > 0. For the second unit: x1 * 1 + x2 * 0 + 1 > 0 (0, because the ...
  • 11
0 votes
1 answer
15 views

Force network to weigh specific variables during learning

I have a pandas data frame containing around 100000 observations of plant species and their age with additional numerical predictors (climate). I used tensorflow ...
0 votes
0 answers
4 views

Is a conv transpose layer equivalent to a padding layer and regular conv layer

Is a 2d convolution transpose layer equivalent to a upsampling layer that inserts 0s between rows and columns, then a regular 2d convolution layer? If so, why is it usually not implemented as such (i....
  • 101
0 votes
0 answers
12 views

What models are able to handle variable input lengths?

I am tasked with creating a classifier that is able to predict whether an item will be returned. This is supposed to not only happen on the basis of an individual item, but on all other items within ...
0 votes
0 answers
18 views

Need help with improving validation loss and model overfitting/underfitting

I am using Ensemble PyTorch to train a voting classifier. My dataset includes around 60k records. I trained a Neural Network with Cross-entropy loss. Below is my model architecture ...
0 votes
0 answers
10 views

Back Propagation on arbitrary depth network with ReLu

I am implementing a neural network of arbitrary depth with an arbitrary number of nodes on each depth. My forwards propagation thus looks like this (For 2 hidden layers) ...
0 votes
0 answers
16 views

Teaching the model on more than one dataframe/dataset

I am trying to write a thesis on oil pipe leakage detection. The aim is to predict the size and location of the leak. My problem is that I can only run single simulations using a software called OLGA ...
0 votes
1 answer
19 views

What does Codex take as tokens?

The typical default for neural networks in natural language processing has been to take words as tokens. OpenAI Codex is based on GPT-3, but also deals with source code. For source code in general, ...
  • 115
1 vote
0 answers
12 views

In WGAN paper, why does clipping weights approximate Lipschitz function?

In Wasserstein GAN, it's explained that maximizing a certain formula over a set of K-Lipschitz functions approximates the 1-Wasserstein distance and they model the functions as NNs. That much I ...
  • 11
2 votes
1 answer
15 views

Im looking for good neurons silmilarity metric

Recently I managed to create simple neural network visualization, to help to understand how neural network works on the signal level. I also wanted to arrange neurons by similarity cause I was ...
0 votes
0 answers
6 views

Optimize wordembedding and neural network at the same time

I have a lot of (domain)-specific text that I want to classify into 100+ categories. I want to train a wordembedding (FastText) and use that in conjuction with a CNN, thus I'm running into the problem ...
0 votes
0 answers
10 views

How do i calculate 1 iteration of the backpropagation algorithm on this exercise

Hello, i'm currently learning Neural Networks, so have a lot of doubts on how do i calculate the weights after one iteration. The step function for neuron 1 is:y1 = step(x1 + x2 -1.5) The step ...
0 votes
0 answers
10 views

Is there a way to use CNN to separate/cluster the images into N clusters without online learning or only very mild online learning?

There are lots of examples to use CNN as classifier to separate images into known classes. There also lots of examples to use CNN as encoder and generate embedding to check the similarity of objects. ...
  • 101
-1 votes
1 answer
29 views

Classifying the sum of two inputs as even or odd - Model always stuck at 50% accuracy for training and test data

I am creating a simple feed forward to classify if the sum of two inputs as even or odd. I cannot change the input structure (has to be two nodes), and output structure (two nodes as well, one for ...
0 votes
0 answers
9 views

I am creating an multilayer LSTM model from scratch and everything seems to be mathematically correct however the model refuses to learn

I am creating the LSTM with just numpy and plotting the loss with pyplot. I have checked the derivatives again and again however have not found a mistake. The entire code with the main function can be ...
0 votes
0 answers
14 views

Combining 2 losses for 2 different tasks and training the networks in Keras

I have to implement a communication model consisting of a mapper , channel , detector and demapper blocks. The mapper , detector and demapper blocks are neural networks and channel is a AWGN channel. <...
0 votes
1 answer
21 views

Machine learning / statistical model of a deterministic process: how large must my training set be to ensure almost perfect accuracy?

This may be a silly question, but if I got a deterministic process, for instance, a function (in the mathematical sense) that happens to be computationally expensive to evaluate, and I decided to ...
  • 103
0 votes
1 answer
30 views

Classification of a noisy data

What method can be used to classify data in the following example? There is a table (hundreds of strings and hundreds of columns). Several columns in this table uniquely allow you to classify each row:...
  • 1
0 votes
0 answers
11 views

How to shape the input for Temporal Convolutional Networks

Consider a normal time series coming from stock prices, assume for simplicity it's several thousand data points. So basically I have a time series of prices $\{x_i\}_{i=0}^n$ of $n$ data points. I ...
  • 101
0 votes
0 answers
11 views

Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?

I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it ...
0 votes
0 answers
16 views

Understanding perceptron learning algorithm

I was revisiting perceptron learning algorithm. The wikipedia page gives the algorithm as follows: Initialize the weights to 0 or a small random value. For each example $j$ in our training set $D$, ...
  • 99
0 votes
0 answers
12 views

CNN sharing weights in feature map

what do they mean when they say all neurons in a channel share weights with one another? Do they mean that in a chanel or a featue map the weights are the same ?
0 votes
0 answers
15 views

Any guidance on designing neural network to fit multiplication?

I want to explore the ability of NN to fit multiplication, so I design a dataset and fit it using LSTM. The dataset: X: samples x 60 x 1; the timestep is 60, the feature dimension is 1 Y: y[i] = corr(...
0 votes
1 answer
21 views

Can I change the number of inputs to a keras model while preserving the trained existing weights

I have a simple Sequential keras model with 150 Inputs. Some of these are simply OneHotEncoded values. Now I would like to add more options to the OneHotEncoder. As an example: I previously had Blue, ...
0 votes
0 answers
17 views

How to visualise a feature map or filter in a group equivariant convolutional neural network

I'm reading an article called "Group Equivariant Convolutional Networks" by T. Cohen and M. Welling (https://arxiv.org/abs/1602.07576) and I'm having some problems understanding one of their ...
0 votes
1 answer
15 views

Train/val/test approach for hyperparameter tuning

When looking to train a model, does it make sense to have a 60-20-20 train val test split, first hyper parameter tuning over the training dataset, using the validation set, picking the best model. ...
  • 91
0 votes
0 answers
16 views

What is semi-ground truth?

What is semi-ground truth and how it is different from ground truth? Can you explain with an example?
  • 1
1 vote
1 answer
20 views

How to determine which combinations of parameters to include in GridSearchCV

I am using MLPClassifier from sklearn and I would like to tune it with GridSearchCV. But I don't know which set of values to include for hidden_layer_sizes, max_iter, activation, solver, etc. How can ...
  • 11
0 votes
0 answers
25 views

Is this overfitting? (generative model)

I am working with a generative method, and the network seems to perform well on training data and slightly less well on test data, but the generated data is somehow significantly worse than either of ...
  • 31
0 votes
0 answers
15 views

Neural networks and input filters

In my use case scenario, I have a neural network that should filter the input and pass a specific value of the input array to the output. In particular, let's define the input as: ...
  • 1
0 votes
0 answers
9 views

OR gate perceptron in plain python - Loss won't converge

I am coding a perceptron from scratch just out of curiosity in plain python for OR gate, but a loss won't converge. ...
0 votes
0 answers
10 views

Weighting loss functions for multi task learning

I am training a multi-task neural network which is predicting a binary target variable, an 18-class target, and a 17-class target. I am calculating the cross-entropy loss for each task, then summing ...
  • 1
0 votes
0 answers
12 views

Relative changes instead absolute values in LSTM training

I am reading some papers about glucose time series prediction and I have noticed that some of them propose LSTM models that use relative changes between two measures. For example, if $$ glucose(t)=60, ...

1
2 3 4 5
86