Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Input size vs hidden state in RNNs

Im using PyTorch to implement RNNs on univariate time series data. This is the documentation for the RNN class: link I think I'm understanding the math behind an RNN cell. But I have an specific ...
  • 101
0 votes
1 answer
22 views

How do I know that my weights optimizer have found the best weights?

I am new to deep learning and my understanding of how optimizers work might be slightly off. Also, sorry for a third-grader quality of images. For example if we have simple task our loss to weight ...
  • 3
1 vote
1 answer
18 views

Should I annotate additional information besides the categories I already need in a text?

I have a dataset with bank transfer reasons. They vary a lot because humans wrote them. From the reasons that are linked to invoice payments I need to extract several things: invoice number(s) IBAN ...
  • 155
-1 votes
0 answers
9 views

More dense layers with heavy dropouts or fewer layers with light dropouts?

I'm trying to build a network. While creating the fully connected part in the last, Which one should we prefer: More layers that regularly reduce with heavy dropouts or fewer layers that reduce ...
-1 votes
0 answers
22 views

Graph Execution Error when dealing with ordinal encoded data

While running a neural network where one of the features was encoded using ordinal encoding, the unknown values in the test data was handled by filling in as -1 using the unknown_value parameter. ...
  • 1
0 votes
0 answers
17 views

Which algorithm to use for predictors which are sparse for classification problem

I have a classification problem with target has 85% to 15 % ratio (0,1) and around 35 predictor which all are 0 or 1 , I tried building logistic regression however the auc is around 0.53 , I am not ...
0 votes
0 answers
10 views

How does posterior distribution relate to model parameters

I want to know how the estimation of a posterior through a sampling method for example HMC could help with predicting a model, and why predicting model weights/parameter is important and how it is so?
  • 53
0 votes
1 answer
31 views

Machine learning algorithms for tabular dataset

I have a dataset with 120 features and 5000 instances. The dataset is combination of categorical and numerical values. It is a tabular dataset. My problem is a binary classification problem. I trained ...
  • 309
0 votes
0 answers
20 views

Unstructured Text Prediction

I have a question regarding a real-world problem on medical data. I have an input and output dataset which looks like this: Input Output Patient is sick with fever Prescribe Panadol and tylenol ...
0 votes
1 answer
21 views

Linear Regression in Pytorch-vanishing gradient with Softmax

I am implementing a non-linear regression using neural networks with one single layer in Pytorch. However, using an activation function as ReLu or Softmax, the loss gets stuck, the value does not ...
0 votes
1 answer
38 views

How do transformers differ from feature selection and regular machine learning?

This is perhaps a simplistic way of thinking, but to me transformers (attention based neural networks) focus on a subset of the input, learning what is important for the problem/prediction as the ...
  • 71
0 votes
0 answers
14 views

Question about LSTM input

I am trying to use LSTM to predict user input but my question is how can you get the actual input of the user and let the LSTM predict it? I tried to check online but I dont see anything about it. I ...
2 votes
1 answer
29 views

Question about non linearity of activation function

I have a basic question about activation functions. It is told that they are added to the network to introduce non linearity. However, the neural network itself is non linear. Isn' it? If we see any ...
0 votes
1 answer
15 views

How to optimally visualise hyperparameter tuning?

I am working on a basic Neural network and want to show performance of model with respect to different parameters. I need help with suggestions for impactful visualizations. I can not make ...
0 votes
0 answers
15 views

Proof that averaging weights is equal to averaging gradients (FedSGD vs FedAvg)

The first paper of Federated Learning "Communication-Efficient Learning of Deep Networks from Decentralized Data" presents FedSGD and FedAvg. In Federated Learning the learning task is ...
1 vote
3 answers
82 views

How to learn steep functions using neural network?

I am trying to use a neural network to learn the below function. In total, I have 25 features and 19 outputs. The above image shows the distribution of two features with respect to one of the outputs....
  • 61
0 votes
0 answers
13 views

Generative Autoencoder with latent vector size as a parameter?

I am interested in using a generative autoencoder (something like a VAE maybe) to sample very high dimensional data more easily (making use of the fact that the autoencoder reduces the dimensionality ...
0 votes
0 answers
12 views

LSTM focus more on long-term memory

I have a specific use case where instead of learning the best LSTM model (least mse), I want a model which has more focus on long term memory (Esentially less decay in LSTM gates). Which hyperparamets ...
1 vote
1 answer
18 views

Why there is so few research on neural code of artificial neural networks and are there alternatives to the neural code approach?

I feel that the neural code/neural coding (how neurons or biases enode the symbolic concepts of the chains of concepts, e.g. each feature is chain of symbolic functions and their parameters) is the ...
  • 131
0 votes
0 answers
17 views

Inconsistency between training and prediction

While training (my 3d_conv NN), the accuracy of the train set is high - 65% (don't mind the validation accuracy yet): But - when I predict the labels of the TRAIN set on the trained model, I get a ...
  • 101
1 vote
1 answer
29 views

Is it possible to attribute reasons for poor (anomalous) target performance?

I am new to this forum and to field of machine learning. My apologies if this is a trivial question already answered previously. I have a set of approximately 20 servers. Each server's performance is ...
0 votes
0 answers
7 views

How to set max_length for bert_preprocess from tf_hub?

I am building a simple BERT model for text classification, using the tensorflow hub. ...
  • 289
2 votes
0 answers
19 views

Neural net patience moving average

To my understanding, when we use patience = 8 when training a neural net, if there is no improvement on the loss (usually the validation set loss) for 8 epochs, then we would stop the training and ...
  • 21
2 votes
1 answer
42 views

How to remove noise from signals?

I have sensor which outputs signals (two signals bellow for example). I use 2000 signals as my data, which some of them are clear and some of them are bad signals. All clear signals have peaks, and ...
0 votes
0 answers
14 views

Time series prediction with feedback

I have a multi-dimensional data, but it misses target variable. I want a model to predict a value for this data that could be passed to my loss function which can be then used as a feedback for ...
0 votes
0 answers
16 views

how to reduce the loss and improve the gradient flow - CNN

I am trying to improve this situation, in image classification[3 classes, softmax in the last layer], I constructed the neural network having 7[conv2d+Batchnormalization] layers + 1 linear layer, ...
1 vote
0 answers
16 views

How to optimize transposed convolution?

I have implemented an encoder-decoder architecture-based neural network with Neural Network API(NNAPI) Android Ndk. There are 5 encoders and 5 decoders. The first encoder's input dimension is -> ...
1 vote
1 answer
22 views

In cross validation, should the test dataset not be fixed

Would this work? We want to train a neural net. We have 50 datapoints and want a split of 30 for train, 10 for validation, 10 for test. We want to do 5-fold cross validation. We use the following ...
  • 11
1 vote
1 answer
24 views

Modelling Player Impact on English Premier League

Let's say you have a very wide feature vector, all 0s and 1s, all 500 players in Premier League as features, out of which only 22 participate in a match. Those that participate are marked with 1s, ...
2 votes
1 answer
23 views

Ignore or predict padding

I have a sequence to sequence classification model with two classes (similar to NER transformer) and because my data samples have different lengths I use padding. Is it better to use a custom loss ...
0 votes
0 answers
8 views

Can neural networks help in Cahn-Hilliard PDE Inpainting?

I am wondering why I have not seen any publication in literature either advocating for or attempting to utilize neural nets to do image inpainting. Yet, PDE (like the Cahn-Hilliard especially) ...
  • 1
0 votes
0 answers
9 views

Best layout for performing CNN with multiple images

I'm looking to create a CNN (probably using ResNet) for automating image rating of videos by processing a predetermined number of frames from the video (60 for example evenly distributed through the ...
1 vote
1 answer
27 views

Recognize chatbox on game screenshots

I have videos from a computer game. In this computer game, during the rounds, there is a chat box where players can write messages. I want to read the content of this chatbox. Difficulties are here: ...
0 votes
0 answers
11 views

Recent research on solving "inverse" ODE problems with neural networks?

I come from a physics background, and I am not familiar with the state-of-the-art research in solving ODE optimization problems with NNs. Let me briefly introduce this so-called "inverse" ...
0 votes
0 answers
13 views

What do filter numbers signify in a convolutional neural network?

I'm new to deep learning and currently trying to figure out the basic concepts about convolutional neural networks. I understand how such a neural network processes images, but there is one thing that ...
  • 1
3 votes
1 answer
43 views

Should I use validation data and val_loss when training final model?

I am training a keras model that utilizes early_stopping in order to prevent overfitting. This requires that I set aside a validation dataset. My task requires that ...
  • 133
0 votes
1 answer
110 views

progress bar for GridSearchCV

I am building CNN algorithm that will output some values. I am using GridSearchCV for parameters tuning and I want to implement progress bar for handling with large datasets but I do not know how to. ...
0 votes
1 answer
46 views

TypeError: object of type 'NoneType' has no len() when implementing neural network

I am building artificial neuron network (ANN) model for predicting values but facing problem: Input: ...
0 votes
0 answers
8 views

Neural network training for multi time scale (fast and slow) data

I have a background in dynamical system control and I am new to machine learning field. In control, we sometimes have a system that has multi time scale dynamics, e.g., some states evolve much faster ...
  • 1
0 votes
0 answers
5 views

Randomisation of contrast / brightness in specialised object detection

I am currently training a object localisation and classification model to find and classify impurities and damages of a polished surface. The analysis is performed on the basis of multiple image which ...
  • 11
0 votes
1 answer
19 views

ML/DL model needed to perform binary classification on binary input image dataset

I desperately need help regarding ML/NN models that would be appropriate for binary input data.. So, I have an image dataset in which [R,G,B] values can only take ...
0 votes
0 answers
15 views

Is it possible to apply graph neural networks on a sequence to determine a hidden states of its elements (instead of HMM)?

For example, I have a sequence of observations from {$y_1$, ..., $y_{4}$}: $y_1, y_2, y_1, y_3, y_2, y_2, y_2, y_4 $ And the sequence is produced by a hidden states from {$X1$, $X2$, $X3$}. Usually ...
0 votes
1 answer
10 views

Difficulty loading data/running model on custom dataset derrived from DNA sequence data - TypeError when attempting to run model

I am a student who has some limited experience with keras, and for a new project recently decided to learn how to use pytorch to implement my models. I'm a beginner with both, so apologies in advance ...
0 votes
0 answers
10 views

tensorflow is not learning

I am writing this code from the tensorflow tutorial about Autoencoders ...
0 votes
0 answers
4 views

How to create a representative dataset for Object Detection?

Consider that we are building a dataset for a network to localize and classify roadsigns. Furthermore, let's say we only care to classify two very similar types of road signs, a 30 and 40 mph speed ...
0 votes
1 answer
19 views

Is it ok if I use early callbacks with restore best weights?

Does anyone know, if it is ok if I use early callbacks with restore best weights? The metric measured by the early callback is validation loss. I was afraid that if I restore the best weights of the ...
0 votes
0 answers
19 views

Practical application of denoising autoencoders

I have been reading into autoencoders for the purpose of denoising data. In the examples i found (eg. [1, 2, 3], which are the first few google results) they have the following input/output: Input ...
0 votes
0 answers
30 views

Validation loss stays bouncing while training loss converges immediately

I'm using bi-GRU to try solving a bi-classification problem. What I have observed is that no matter how much dropout(from 0 to 0.6) and layer-norm I added, the training process shows similar situation:...
  • 1
0 votes
0 answers
23 views

Classification: ClassA vs. "everything else"

I am trying to create a neural network for recognizing a particular object. Maybe I am approaching this task from the wrong side, but, in my mind, this task boils down to teaching the network to do a ...
0 votes
1 answer
15 views

Plot NN regression/similar predicted output for all inputs

Can someone explain why the Fit line does not match the true label? I do not understand what exactly is happening and where I have a problem in my model. Here is ...

1
2 3 4 5
84