Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

Should rescaling be used on test images in keras?

I am kind of confused regarding the topic. I have built a CNN architecture for the cat-dog image classification around 6000 images of cat and 6000 images of dog and I am predicting on test images. I ...
0
votes
0answers
29 views

Long term mathematical side effects of overfitting a model to reinforce a behavior

I have a neural network I designed, and previously I got this network working with some considerable simplification to its design which I was not fond of making. However I noted that during training ...
1
vote
0answers
11 views

Is it possible to see the output for each layer in a mlp? using SKLearn

I have a simple NN which I have made with SKLearn. I have extracted: The weights sent to each node The bias assigned to each activation function But I can't see a way to get the output of the ...
3
votes
0answers
20 views

Understanding an MLP coefficient array

I have implemented a super simple MLP using SKLearn. I have a 2 hidden layer model and 31 features on the input layer. So the lengths of the arays are 31, 20 and 10. ...
3
votes
1answer
29 views

Multi-modal neural network

How do you start with creating a multi-modal neural network?
2
votes
0answers
33 views

Cross Validation in Neural Networks

I am training a neural network and doing 10-fold cross validation to measure performance. I have read lots of documentation and forums telling that the set of weights that should be saved or ...
1
vote
0answers
15 views

How to load a dataset with a specific structure in tfds library?

I have a dataset that it's classes arranged in the following way: /dataset/train/images/class1/ /dataset/train/images/class2/ . . . /dataset/train/images/classN/ ...
0
votes
1answer
27 views

Improving the performance of neural networks

I have 3 questions in mind about the neural network For the best model performance, is it better to train a model only on high resolution images or does it not matter whether the training data ...
0
votes
0answers
30 views

Improve model accuracy in multi-classification problem

I use a MLP to classify three different classes A, B, C. The loss function I use is categorical cross entropy and the optimiser ...
2
votes
1answer
33 views

Why do Transformers need positional encodings?

At least in the first self-attention layer in the encoder, inputs have a correspondence with outputs, I have the following questions. Isn't ordering already implicitly captured by the query vectors, ...
0
votes
0answers
9 views

Neural network training time

I am very new to NN and I have a dataset that I use NN to map the relation between its inputs and output using MATLAB and I will test different NN architectures and algorithms. Regarding training time,...
2
votes
2answers
31 views

Deep Neural Network Model in sklearn Pipeline

Is it possible to add a deep neural network model as the estimator/model in an sklearn Pipeline? or is it only possible for ML models as the estimator. For example, ...
0
votes
0answers
15 views

How to decide number of hidden layers and number of neurons for Autoencoder for dimensionality reduction function?

I have been looking into deep learning and what caught my attention is the implementation of Autoencoder as a dimensionality reduction function for anomaly detection. I found out about it through the ...
0
votes
1answer
16 views

If we train a model every time from scratch by using current task and samples from memory (ER) then is it correct way to perform continual learning?

Suppose that there are T tasks. We use an experience replay (ER) strategy using a tiny episodic memory. Here, we train a model always from scratch at each task using current task samples and samples ...
0
votes
0answers
8 views

Assess feature importance in Keras for one-hot-encoded categorical features

An important aspect of tuning a model is assessing feature importance. In Keras, how to assess the importance of a categorical feature which is one-hot encoded? E.g. if a categorical feature is ...
-1
votes
0answers
15 views

Better Confidence Index for tracking neural network progress [closed]

In this question I indicate a method by which I am trying to get a feel for how correct the values are being generated by the neural network. Training seems to be plateauing at every learning rate I ...
0
votes
0answers
20 views

Improving validation losses and accuracy for 3D CNN

I have used a 3D CNN architecture, for detecting the presence of a particular promoter (MGMT), by using FLAIR brain scans. (64 slices per patient). The output is supposed to be binary (0/1). I have ...
0
votes
0answers
26 views

Training seems to be plateauing at every learning rate

So firstly I have a network that I'm using to approximate the value of a function. Recently, at about 50000 trains, it began to show no further advancement in training, at any learning rate. The ...
2
votes
0answers
8 views

Heterogenous data between classes in object detector

I have created a synthetic dataset for object detection with about a dozen classes by placing models of the classes in front of random images (from the MIT Place2 dataset) in Blender. This is working ...
0
votes
0answers
26 views

Does having two different models improve performance for underrepresented classes?

I am currently working on a dataset that has approximately 7000 annotations, but suffers from severe class imbalance (there are 1331 annotations for the most represented class, and 77 for the least ...
0
votes
0answers
25 views

How to use Softmax and CrossEntropyLoss in a classification problem?

I am new in the deep learning field and I would like to go in to understand some concepts. I started learn with pytorch and I have the next question: I constructed my own dataset that contains images ...
1
vote
0answers
19 views

Should I reshuffle the training set when benchmarking neural networks?

I'm trying to set up a fair benchmarking between various RNN models, where each of them is trained until convergence with a fixed random seed. Because the task is very costly, I am only able to run ...
2
votes
0answers
20 views

Classifying short strings of text with additional context

I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match. ...
0
votes
0answers
10 views

How to represent source code in NLP for tasks like code retrieval?

How to represent code features in NLP? Can code be treated like a language and pour into the neural network? Are there some work, paper or material around this topic?
0
votes
0answers
15 views

How can I create an XOR Gate using two hidden layers and one output layer (besides the or gate)?

Here is my nueron class with activation function (sigmoid), fn for printing truth table and inputs Mathematically and_gate + nor_gate + nor_gate are giving me the correct output. But the function is ...
0
votes
1answer
17 views

Segmentation 3D Unet checkerboard artifacts in slices above and below segmentation voxels

I suppose an image is worth too many words, so here is the image: As you can see, in the middle where there are voxels to be segmented, no artifacts are present. Whereas on the top and bottom I get a ...
0
votes
0answers
22 views

Where is this Pytorch NN version of a Keras example wrong?

I want to write a network inverting the Radon transform. I found an example in Keras here. The network is given by: ...
1
vote
1answer
18 views

Understanding deconvolutional network loss function

In the paper (1), there is a description of a deconvolutional network. The loss function (with only one layer) compares the colour channels of the orignal image with the colour channels of the ...
0
votes
0answers
6 views

How to train a neural network to provide minor formatting edits to words in a glossary?

I would like to try to train a neural network to recognize basic formatting issues in a glossary. This could include that a word is in the plural rather than the singular, that it begins with a ...
0
votes
0answers
11 views

Input Neural Network

I have often seen something like this: tf.placeholder(dtype=tf.float32, shape=[None, input_dim]) as an input for a neural network. But the only input that makes ...
7
votes
1answer
100 views

Why deep learning models still use RELU instead of SELU, as their activation function?

I am a trying to understand the SELU activation function and I was wondering why deep learning practitioners keep using RELU, with all its issues, instead of SELU, which enables a neural network to ...
0
votes
0answers
21 views

How to determine a good architecture for multilabel classification

I am working on an university project that requests us to classify Wikipedia abstracts about people by their professions. The output shall be a JSON file that assigns each Wikipedia abstract to a set ...
0
votes
1answer
15 views

predict a binary vector of size 40

I have a dataset of shape (2600, 95) with first 55 columns are features and 40 columns are label. Label is a binary matrix of size 10x4 that flattened, and features are real valued numbers ranging (0....
0
votes
0answers
17 views

Can quantitative data be calculated from a neural network?

The only success I've seen or had with Neural Nets, is taking whatever input, and outputting Boolean results, yes/no, in the form of a range between in the case in question $0.5$ and $1.0$, with $0&...
1
vote
1answer
15 views

Explanation of Karpathy tweet about common mistakes. #5: "you didn't use bias=False for your Linear/Conv2d layer when using BatchNorm"

I recently found this twitter thread from Andrej Karpathy. In it he states a few common mistakes during the development of a neural network. you didn't try to overfit a single batch first. you forgot ...
0
votes
1answer
25 views

LSTM layer (keras) is causing all layers after it to constantly predict the same thing no matter the input

I have a model for OCR, which after 2-3 epochs gives the same output. When I predicted the values and looked at the output for each layer I realized that all layers after the 1st layer in the LSTM ...
0
votes
0answers
6 views

Levenberg-Marquardt vs Adam optimizer

I am working on a NN and I have to decide between Levenberg-Marquardt and adam optimizer. Adam Optimizer is relatively the newer algorithm and is quite popular. Which optimizer is the better of the ...
1
vote
1answer
17 views

Difference in result in every run of Neural network?

I have written a simple neural network (MLP Regressor), to fit simple data frame columns. To have an optimum architecture, I also defined it as a function to see whether it is converging to a pattern. ...
0
votes
0answers
8 views

Training the input to a neural network

I have a network that takes as input 3 numbers and outputs 2 and I trained it on a given data set, to predict the 2 numbers. Now I would like to freeze all the layers of the network, and make the ...
0
votes
0answers
36 views

nn.embedding alternative for float numbers

I have found this pytorch code of transformers suitable for machine translation: ...
1
vote
1answer
14 views

Pre-processing images for fine-tuning

When you are fine-tuning a CNN like ResNet, VGG, EfficientNet, etc and you want to train the model with your own images, or even when you want to do a inference with any image of your dataset, do you ...
0
votes
2answers
64 views

How does Gradient Descent work? [duplicate]

I know the calculus and the famous hill and valley analogy (so to say) of gradient descent. However, I find the update rule of the weights and biases quite terrible. Let's say we have a couple of ...
0
votes
0answers
15 views

Differentiation of Learning Capabilities of Different Networks

I have a conceptual problem regarding the overall learning capability of a neural network differentiated by the different types of input that we can give to the network. Suppose that we have a ...
1
vote
1answer
33 views

Difference between ReLU, ELU and Leaky ReLU. Their pros and cons majorly

I am unable to understand when to use ReLU, ELU and Leaky ReLU. How do they compare to other activation functions(like the sigmoid and the tanh) and their pros and cons.
0
votes
0answers
37 views

Is there any mathematical basis for distributed training of a neural network on multiple machines?

Let us say that I have 50 people download a program that engages in supervised training of a neural network I designed. The data samples being provided are pretty random and the correct outputs are ...
0
votes
0answers
7 views

Novice machine learner wondering how to interpret big variance in batch error across batches in MNIST perceptron

I'm trying to get a better understanding of basic neural networks by implementing a little framework in C++. I've started with the classical MNIST exercise. I get to 91% accuracy on the test sample ...
0
votes
0answers
13 views

Why does my training explode?

I'm trying to implement A new lower and upper bound estimation model using gradient descend training method for wind speed interval prediction For simplicity purposes, I've changed the training data. ...
0
votes
0answers
8 views

How to do Interpretable AI with multiple image input fields and a single real-valued output image?

I have input data of the shape: (num_ex, num_features, 64, 64) and real-valued output data of shape: (num_ex, 1, 64, 64) I ...
2
votes
0answers
19 views

How to model prior informaton in sequential models?

Are there any approaches to model prior information in sequential models? Such as in Sequence classification. For example, I have an input sequence [[Z, 0, 1], [Y, 1, 1]]. I need to classfy this into ...

1
2 3 4 5
79