Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

transforms in mathematical equations

How to write transform model in mathematical equations, from the start to the end, that's include the formula of y^. instead of coding and visualizations I'm looking for the mathematical equations, as ...
0
votes
1answer
59 views

How could you build a Neural Network with the same weights for different nodes in the same layer

The structure I'm imagining is like the one in the image bellow where the output is softmax. For the hidden layer we would have $$Z_1 = W_{11}^{[1]}x_2 + W_{12}^{[1]}x_2$$ $$Z_2 = W_{21}^{[1]}x_2 + ...
0
votes
0answers
33 views

Random predictions from a multi-label image classification model

I am currently trying to create a model for multi-label image classification using Keras. The model has 8 classes, with a slight imbalance in the dataset (some have 300~, whereas some others have 600~)...
0
votes
0answers
18 views

Why is the signed gradient of the image used for adversarial examples [duplicate]

In this paper, the gradient of the loss w.r.t. to the image is computed, but its sign is used. Why is using the sign-method better?
1
vote
0answers
18 views

CNN-LSTM for price estimation

I have some pictures that I would like to extract features from using ResNet, and then apply another model like LSTM with extracted features to estimate the price. I have made two folders, one ...
0
votes
0answers
26 views

Understanding data and regularization loss

I am reading & coding along Neural Networks from scratch in Python. In it there is a section about L1 / L2 regularization ...
3
votes
1answer
134 views

What changes is the Neural Network back-propagation algorithm doing on the weights?

I have seen the formula for back-propagation algorithm for neural network error minimization, but I am not quite sure about what changes it is performing on the weights individually. Let us suppose a ...
0
votes
1answer
20 views

Understanding Max Pooling

I understand max pooling in CNNs can help decrease the computational load due to downsampling. Another thing mentioned is that max pooling can help provide a sort of "spatial invariance" for ...
0
votes
0answers
29 views

How to compute gradient of Softmax in backward pass of a neural network?

I am trying to implement the backward pass of a Softmax layer. As an input to my backward-function, I receive the gradient from the next upper layer and I have to ...
0
votes
0answers
10 views

Structural Equation Modeling as Neural Network

Is it possible to create an architecture Structural Equation Modeling as Neural Network ?
0
votes
0answers
9 views

Multi-class classification but a single feature sometimes boils it down to a binary-classification

I have a three-class classification problem for a large dataset. Classes are 0, 1 and 2. There's a categorical variable in my feature vectors such that when a sample point has this variable positive, ...
1
vote
0answers
28 views

How to use Lat/Lon in machine learning prediction?

I am research meteorologist working with hurricane model forecasts of track (millions of lat/lon pairs) and their verified lat/lon pairs (essentially where the hurricane actually went). With the ...
0
votes
0answers
58 views
0
votes
1answer
63 views

Should rescaling be used on test images in keras?

I am kind of confused regarding the topic. I have built a CNN architecture for the cat-dog image classification around 6000 images of cat and 6000 images of dog and I am predicting on test images. I ...
1
vote
0answers
46 views

Long term mathematical side effects of overfitting a model to reinforce a behavior

I have a neural network I designed, and previously I got this network working with some considerable simplification to its design which I was not fond of making. However I noted that during training ...
0
votes
0answers
13 views

Is it possible to see the output for each layer in a mlp? using SKLearn

I have a simple NN which I have made with SKLearn. I have extracted: The weights sent to each node The bias assigned to each activation function But I can't see a way to get the output of the ...
2
votes
0answers
30 views

Understanding an MLP coefficient array

I have implemented a super simple MLP using SKLearn. I have a 2 hidden layer model and 31 features on the input layer. So the lengths of the arays are 31, 20 and 10. ...
2
votes
1answer
32 views

Multi-modal neural network [closed]

How do you start with creating a multi-modal neural network?
1
vote
0answers
42 views

Cross Validation in Neural Networks

I am training a neural network and doing 10-fold cross validation to measure performance. I have read lots of documentation and forums telling that the set of weights that should be saved or ...
1
vote
1answer
32 views

How to load a dataset with a specific structure in tfds library?

I have a dataset that it's classes arranged in the following way: /dataset/train/images/class1/ /dataset/train/images/class2/ . . . /dataset/train/images/classN/ ...
0
votes
1answer
31 views

Improving the performance of neural networks

I have 3 questions in mind about the neural network For the best model performance, is it better to train a model only on high resolution images or does it not matter whether the training data ...
0
votes
0answers
36 views

Improve model accuracy in multi-classification problem

I use a MLP to classify three different classes A, B, C. The loss function I use is categorical cross entropy and the optimiser ...
2
votes
1answer
45 views

Why do Transformers need positional encodings?

At least in the first self-attention layer in the encoder, inputs have a correspondence with outputs, I have the following questions. Isn't ordering already implicitly captured by the query vectors, ...
0
votes
0answers
9 views

Neural network training time

I am very new to NN and I have a dataset that I use NN to map the relation between its inputs and output using MATLAB and I will test different NN architectures and algorithms. Regarding training time,...
1
vote
2answers
59 views

Deep Neural Network Model in sklearn Pipeline

Is it possible to add a deep neural network model as the estimator/model in an sklearn Pipeline? or is it only possible for ML models as the estimator. For example, ...
0
votes
0answers
25 views

How to decide number of hidden layers and number of neurons for Autoencoder for dimensionality reduction function?

I have been looking into deep learning and what caught my attention is the implementation of Autoencoder as a dimensionality reduction function for anomaly detection. I found out about it through the ...
0
votes
1answer
18 views

If we train a model every time from scratch by using current task and samples from memory (ER) then is it correct way to perform continual learning?

Suppose that there are T tasks. We use an experience replay (ER) strategy using a tiny episodic memory. Here, we train a model always from scratch at each task using current task samples and samples ...
0
votes
0answers
29 views

Assess feature importance in Keras for one-hot-encoded categorical features

An important aspect of tuning a model is assessing feature importance. In Keras, how to assess the importance of a categorical feature which is one-hot encoded? E.g. if a categorical feature is ...
0
votes
1answer
24 views

Improving validation losses and accuracy for 3D CNN

I have used a 3D CNN architecture, for detecting the presence of a particular promoter (MGMT), by using FLAIR brain scans. (64 slices per patient). The output is supposed to be binary (0/1). I have ...
0
votes
0answers
27 views

Training seems to be plateauing at every learning rate

So firstly I have a network that I'm using to approximate the value of a function. Recently, at about 50000 trains, it began to show no further advancement in training, at any learning rate. The ...
2
votes
0answers
13 views

Heterogenous data between classes in object detector

I have created a synthetic dataset for object detection with about a dozen classes by placing models of the classes in front of random images (from the MIT Place2 dataset) in Blender. This is working ...
0
votes
0answers
30 views

Does having two different models improve performance for underrepresented classes?

I am currently working on a dataset that has approximately 7000 annotations, but suffers from severe class imbalance (there are 1331 annotations for the most represented class, and 77 for the least ...
0
votes
0answers
27 views

How to use Softmax and CrossEntropyLoss in a classification problem?

I am new in the deep learning field and I would like to go in to understand some concepts. I started learn with pytorch and I have the next question: I constructed my own dataset that contains images ...
1
vote
0answers
19 views

Should I reshuffle the training set when benchmarking neural networks?

I'm trying to set up a fair benchmarking between various RNN models, where each of them is trained until convergence with a fixed random seed. Because the task is very costly, I am only able to run ...
2
votes
0answers
22 views

Classifying short strings of text with additional context

I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match. ...
0
votes
0answers
11 views

How to represent source code in NLP for tasks like code retrieval?

How to represent code features in NLP? Can code be treated like a language and pour into the neural network? Are there some work, paper or material around this topic?
0
votes
0answers
18 views

How can I create an XOR Gate using two hidden layers and one output layer (besides the or gate)?

Here is my nueron class with activation function (sigmoid), fn for printing truth table and inputs Mathematically and_gate + nor_gate + nor_gate are giving me the correct output. But the function is ...
0
votes
1answer
51 views

Segmentation 3D Unet checkerboard artifacts in slices above and below segmentation voxels

I suppose an image is worth too many words, so here is the image: As you can see, in the middle where there are voxels to be segmented, no artifacts are present. Whereas on the top and bottom I get a ...
0
votes
0answers
22 views

Where is this Pytorch NN version of a Keras example wrong?

I want to write a network inverting the Radon transform. I found an example in Keras here. The network is given by: ...
1
vote
1answer
20 views

Understanding deconvolutional network loss function

In the paper (1), there is a description of a deconvolutional network. The loss function (with only one layer) compares the colour channels of the orignal image with the colour channels of the ...
0
votes
0answers
6 views

How to train a neural network to provide minor formatting edits to words in a glossary?

I would like to try to train a neural network to recognize basic formatting issues in a glossary. This could include that a word is in the plural rather than the singular, that it begins with a ...
7
votes
2answers
187 views

Why deep learning models still use RELU instead of SELU, as their activation function?

I am a trying to understand the SELU activation function and I was wondering why deep learning practitioners keep using RELU, with all its issues, instead of SELU, which enables a neural network to ...
0
votes
0answers
23 views

How to determine a good architecture for multilabel classification

I am working on an university project that requests us to classify Wikipedia abstracts about people by their professions. The output shall be a JSON file that assigns each Wikipedia abstract to a set ...
0
votes
1answer
20 views

predict a binary vector of size 40

I have a dataset of shape (2600, 95) with first 55 columns are features and 40 columns are label. Label is a binary matrix of size 10x4 that flattened, and features are real valued numbers ranging (0....
0
votes
0answers
17 views

Can quantitative data be calculated from a neural network?

The only success I've seen or had with Neural Nets, is taking whatever input, and outputting Boolean results, yes/no, in the form of a range between in the case in question $0.5$ and $1.0$, with $0&...
1
vote
1answer
18 views

Explanation of Karpathy tweet about common mistakes. #5: "you didn't use bias=False for your Linear/Conv2d layer when using BatchNorm"

I recently found this twitter thread from Andrej Karpathy. In it he states a few common mistakes during the development of a neural network. you didn't try to overfit a single batch first. you forgot ...
0
votes
1answer
44 views

LSTM layer (keras) is causing all layers after it to constantly predict the same thing no matter the input

I have a model for OCR, which after 2-3 epochs gives the same output. When I predicted the values and looked at the output for each layer I realized that all layers after the 1st layer in the LSTM ...
0
votes
0answers
29 views

Levenberg-Marquardt vs Adam optimizer

I am working on a NN and I have to decide between Levenberg-Marquardt and adam optimizer. Adam Optimizer is relatively the newer algorithm and is quite popular. Which optimizer is the better of the ...
1
vote
1answer
19 views

Difference in result in every run of Neural network?

I have written a simple neural network (MLP Regressor), to fit simple data frame columns. To have an optimum architecture, I also defined it as a function to see whether it is converging to a pattern. ...

1 2
3
4 5
80