Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Neural-Networks - preferred method for training, classification v.s. regression

As a conclusion of their paper "Efficient Backprop" (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf) (§10 Discussion and Conclusion), LeCun and others conlude that the preferred method for ...
0
votes
0answers
10 views

Cross layer parameter sharing in ALBERT Model

I am reading the paper "ALBERT: LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS". ALBERT uses cross layer parameter sharing to improve the model performance. I don't understand how ...
1
vote
0answers
16 views

How should i build a Knowledge Graph for a custom dataset?

I'm new to machine learning and i'm trying to create a small Knowledge Graph for search purposes similar to google for a class project. Okay, so i have been searching on this topic for few days and ...
1
vote
0answers
9 views

How to use the model after cross-validation using NN in Matlab

I am following the Matlab implementation of CV with NN. The following code explains training by cross validation (CV) aprroach on fisheriris dataset. Problem1: In this code if you could refer to the ...
-1
votes
0answers
18 views

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB

I'm trying to make a GAN for generating pictures using this code: https://github.com/abhinav3/Udacity-DCGAN-FaceGeneration/blob/master/dlnd_face_generation.ipynb I'm doing this learning on GPU. The ...
2
votes
0answers
11 views

Understanding computations of Percetron and Multi-Layer Perceptrons on Geometric level

I am currently watching amazing Deep Learning lecture series from Carnegie Melllon University, but I am having little bit of trouble understanding how Perceptrons and MLP are making their decisions on ...
3
votes
0answers
25 views

Avoid Load and re-Train of a Neural Network Model [closed]

I built a feedforward neural network, his goal is to classify a target value, the train has to be defined for n-days. The input is constituted by 12 neurons (since I have 12 different data each of ...
1
vote
0answers
20 views

Neural network training returns always the same error rate

I'm playing around with AForge and I want to create a neural network which detects pictures with one point as right and pictures with more points as wrong. I use 10 right and 10 wrong pictures for ...
1
vote
0answers
7 views

How does one use activation function with greater than [-1;1] range for binary classification?

In Efficient Backprop (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf), Lecun and others propose to use activation function that don't reach target values on their asypmptotes. They explain (§ 4....
2
votes
0answers
12 views

How does one derive the modified tanh activation proposed by LeCun?

In "Efficient Backprop" (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf), LeCun and others propose a modified tanh activation function of the form: $$ f(x) = 1.7159 * tanh(\frac{2}{3}*x) $$ ...
1
vote
0answers
31 views

Keypoint detection from an image using a neural network

I am trying to design and train a neural network, which would be able to give me coordinates of certain key points in the image. Dataset I've got a dataset containing 1800 images similar to these: ...
1
vote
0answers
16 views

How to handle imbalanced Data set features for classification problems?

I have a Data set with 300 features that are dummies 0 or 1 and two classes for output 0 or 1 and the distribution of classes is 48% and 52%, but the distribution of the features is skewed as in the ...
1
vote
0answers
25 views

How to estimate and/or determine how much data is enough to train a model?

To train a nice supervised algorithm (for instance, a dependency parser, a parts-of-speech tagger or NER) data is essential, but how many samples are necessary or enough? From what kind of ...
3
votes
1answer
23 views

Is it wrong to use Glorot Initialization with ReLu Activation?

I'm reading that keras' default initialization is glorot_uniform. However, all of the tutorials I see are using relu ...
6
votes
1answer
354 views

How does attention mechanism learn?

I know how to build an attention in neural networks. But I don’t understand how attention layers learn the weights that pay attention to some specific embedding. I have this question because I’m ...
1
vote
0answers
19 views

Tensorflow, Optimizer.apply_gradient: 'NoneType' object has no attribute 'merge_call'

My programme gives the following error message: ...
1
vote
0answers
17 views

Implementing an RNN on multiple text sources

I want to implement an RNN to generate a new text based on many examples of existing texts of a certain format in the training data. The type of texts in the training data consists of 3 segments, ...
2
votes
1answer
22 views

Magnifying or reducing the size of input groups into a neural network

Say you've got two inputs (X1 and X2) that you want to use to predict Y. You're not sure how important X1 and X2 are for predicting Y, but you assume about even. One-hot encoding is a good strategy ...
0
votes
0answers
21 views

Autoencoder for anomaly detection, output layer activation function

I am building an Autoencoder to detect anomalies. I have mixed data, i.e continuous and categorical. I have one-hot encoded the categorical data. Scaled the data with a MinMax scaler. To determine if ...
0
votes
0answers
21 views

Manual Calculation for Backpropagation

I am trying to understand the backpropogation in neural net, Take a look at this below picture Here, I understand the formula for calculating error in Weight5(w5) and also updating the w5. But, I am ...
1
vote
2answers
30 views

h in LSTM increasing in size?

So I was reading about the LSTM architecture and I was having trouble understanding a certain aspect of it. This article mentions the step in question near the bottom of the page. Here is the image ...
0
votes
0answers
14 views

Any one has data set of apple disease detection? [closed]

I am working on Disease detection of Apple fruits so if any one has info please comment me.
1
vote
1answer
12 views

Scalable way to train graph embeddings

I have a use-case to train graph embeddings, looking for a way to do it in pytorch and tensorflow. The restriction is the methodology should incorporate edge weights in calculating transition ...
2
votes
2answers
86 views

What is the role of $W_{ax}, W_{aa}, W_{ay}$ in forward propagation in RNN? Are they hyperparameters? Why are they needed?

In RNN introduction in Coursera sequence model course, the following formula for forward propagation in RNN was introduced. What exactly is the role of $W_{ax}, W_{aa}, W_{ay}$? What do they do? In ...
1
vote
1answer
27 views

Why would one use Deep Learning on non-local datas?

I understand the using of deep-learning for datas that have "local" datas, for example images/videos/texts, as the convolutionals layers reduce the amount of dimensions. However, I saw that some ...
2
votes
1answer
25 views

Interpreting Gradients and Partial Derivatives when training Neural Networks

I am trying to understand of purpose of partial differentiation in NN training by knowing how to interpret gradients and their partial derivatives. Below is my way of interpreting them so I would like ...
3
votes
1answer
13 views

Physical modelling with neural networks - single output + stack ensemble vs multi-output

We are trying to replace an existing physical model (8 inputs/7 outputs) with artificial neural networks. The physics behind the existing model is mainly thermodynamics of humid air for air ...
2
votes
1answer
66 views

How to properly apply CrossValidation and/or split the dataset?

I have a particular problem and do not really now how to properly validate my experiments in this scenario. There is one big data set with 100.000 samples, 99.000 y=0, 1.000 y=1 Each sample has 1.000 ...
1
vote
2answers
19 views

Justification for values used in backpropagation

I'm learning the method for backpropagation in adjusting weights. A generalization of a formula used to determine the change made to a respective weight is where is the rate the total error changes ...
2
votes
1answer
29 views

Why does MAE differ after prediction (Neural Network)?

I'm having trouble understanding what's happening in the following code. I already have defined x_train, y_train, x_val, y_val and x_test which define my training, validation and test sets. I'm using ...
5
votes
1answer
41 views

Can I completely cancel the effects of using a smaller batch size by reducing the learning rate?

I'm having the problem that the data from a regular sized batch (e.g., 32, 64) doesn't fit in my GPU. Among other solutions, I'm considering reducing the batch size, as is normally suggested. Of ...
3
votes
1answer
35 views

Risk prediction vs classification model

I am working on a binary classification model. Currently, when I use scikit logistic regression, it outputs binary values like 0s and 1s. However, I understand, from online reading, that it outputs ...
2
votes
0answers
12 views

How to create anchor-positive and anchor-negative pair for feature X in a signature data set for training Siamese network

How to create anchor-positive and anchor-negative pair for feature X in a signature data set for training Siamese network? Im have a cedar signature data set with 55 peoples signatures(classes) with ...
0
votes
0answers
11 views

goodness of fit metrics to compare neural network and GLM model for count data

I´m wodering if some of you have compared goodness of fit of a NN and a GLM model on count data and which metrics you used? In addition, the data I´m dealing with has a point mass at zero. Are there ...
2
votes
1answer
39 views

Machine learning solution approach to match loan repayments

I'm relatively new to the AI/ML space, but come from a programming background. The problem: I have a dataset of users transactions who have taken short-term loans from a single loan provider and I ...
2
votes
2answers
40 views

Forward pass vs backward pass vs backpropagation

As mentioned in the question, i have some issues understanding what are the differences between those terms. From what i have understood: 1) Forward pass: compute the output of the network given the ...
2
votes
1answer
23 views

Using embedding layer output as input to .fit() call in Keras

I want to build a classifier in Keras that predicts the next item bought by a customer (i.e. multiclass classification). One of the features I intend to input to the model will be the last item bought ...
1
vote
0answers
78 views

Non differentiable loss function

I have a loss function that minimizes the error according to what I want the neural network to do. The problem is, that it is a nondifferentiable function. How can I handle this? the loss function: $(...
2
votes
0answers
17 views

Understanding the significance of LeNet-5 w/ MNIST data set

I'm beginning to learn about conv nets and started with what I understand to be one of the seminal works: LeNet-5. However, my limited experimentation doesn't seem to show any advantage over a single ...
2
votes
2answers
43 views

Drawing Neural Network diagram for academic papers

Is there any tool that one can use to draw neural network architecture diagram for research papers? Example diagram:
2
votes
1answer
29 views

Find Net Reclassification Improvement/Index metric using Python

I am working on a binary classification problem with ~5k records and class proportion of 33:67. I have 60 features/variables in my dataset and finally I have come to about 10 variables based on ...
2
votes
1answer
31 views

How to evaluate performance of a new feature in a model?

I am working on a binary classification where I have 4712 records with Label 1 being 1554 records and Label 0 being 3558 records. When I tried multiple models based on 6,7 and 8 features, I see the ...
1
vote
1answer
28 views

Why results of statsmodel logreg is different from scikit-learn logreg?

I am trying to do a binary classification. I have only 6 input variables and one output variables. Label 1 is 1554 records and Label 0 is 3558 records. As you can see below, the metrics that I get ...
2
votes
1answer
21 views

Extracting name, date and total from a set of heterogeneous receipts

So, this is how the problem goes: I am trying to extract information from scanned receipts like this, What I have been told is that I would get the textual data from a OCR software, so in short I ...
2
votes
1answer
29 views

Unable to do resampling using Python SMOTE

I am trying to do a simple ML re-sampling approach after the train-test split. However when I do this, it throws the below error. Can you please help me understand what is this error about? ...
1
vote
0answers
12 views

Algorithm for EMG dataset

I have an EMG dataset that depicts 6 different gestures depending on the measurements at intervals of roughly 1ms, from 5 electrodes placed equidistant on the wrist. I have the data for 36 different ...
5
votes
1answer
60 views

Machine learning methods for panel (longitudinal) data

I have a panel data set, for example: ...
1
vote
0answers
24 views

Where can I get pre-trained weights for character recognition?

I'm doing a simple pipeline and I need a digit recognition algorithm. There are definitely pre-trained weights out there for character recognition, but I can't find them. They can be in Keras, ...
2
votes
0answers
78 views

Relation of Tensors to artificial neural networks

When dealing with artificial neural networks, one is bound to come across the concept of Tensors. In such applications a Tensor is essentially just a data grid, just like a scalar (rank=0), a vector (...
6
votes
1answer
44 views

Using a GAN discriminator as a standalone classifier

The goal of the discriminator in a GAN is to distinguish between real inputs and inputs synthesized by the generator. Suppose I train a GAN until the generator is good enough to fool the ...